Stephen R. Covey, author of the hugely successful book
"The 7 Habits of Highly Effective People," died July 16th 2012.
I picked up his book because I realized that I need to
nurture my own leadership skills in order to promote product quality at the
next level. Working as an individual contributor doesn't cut it anymore; I need
to influence the big picture, not patch up automation projects that are already
established according to the patterns of software quality as it's generally
practiced today.
Serendipity! There it is, in the first full chapter of his
book. Covey pithily describes his journey of discovery through American self-help
literature as a tension between two approaches toward being personally
effective: The "Character Ethic" and the "Personality
Ethic."
The Character Ethic is deep and foundational, but not always
obvious on the surface behaviors of a person. The Personality Ethic displays on
the surface, but doesn’t necessarily have any depth.
Covey dismisses the Personality Ethic approach to increasing
personal effectiveness as beneficial in some settings, but weak in the long
term and perhaps even damaging, whereas the Character Ethic flows from a
person's basic values and practices.
He describes his own paradigm shift, triggered by a
parenting challenge with one of his sons. Covey used the personality ethic as
taught in his day to guide his parenting style, but he transitioned to the
character ethic (as described by Benjamin Franklin) with good effect. The
Character Ethic requires taking responsibility, and Covey shows that his
paradigm shift is complete when he takes responsibility for his actions re. his
son.
I've seen this happen at many software teams: people do software
quality by emphasizing a basic automation of the product. The result is N test
cases automated, and M of them pass. What happens with the N-M test cases
automated that do not pass is not considered important, except that the team
recognizes the need to get these test cases running again at some point. The
highest priority - I've seen this priority be placed even higher than fixing
"broken" test cases - is automating more test cases. The count of
test cases automated is a superficial measure of success and productivity in
measuring the quality of the product, but it's what they are measured on.
Management needs a metric, and this is the best or most usable one they know
of.
Covey's shift from personality ethic to character ethic is a
pretty good analogy to the shift away from the
automate-as-many-test-cases-as-you-can focus, towards the deep automation that
really tests the product, and attempts to maximize the actionability of test
failures. (The first approach might use a minimal positive-result verification
like this: the automated test didn't throw an exception, therefore it's good.)
Focusing on automated test count– that is, the traditional
way of automating the product - has value in the near term because it really does make the product do stuff and you can find a lot of product bugs through
the process of creating automation. But, in the longer term with this approach,
the inevitable failures of these tests are costly to follow up on and often the
team gets an inaccurate picture of how completely the quality of the product
has been measured because the automated test might hide failures or even be
testing the wrong thing. I've even seen a very expensive 3rd-party test
harness report success on a test, when on detailed follow-up, I found that the
test didn't do anything meaningful.
Test automation with attention to metaautomation takes some
up-front test harness design and more careful test automation with attention to
patterns of failure reporting, but in the longer run a better and more complete
measurement of product quality happens; test failures due to product issues are
more actionable and are likely to be followed up on quickly. Trust in test
automation code is higher, which enables the team to be more productive. Most
importantly: failures due to regressions in the product are fixed more quickly,
which keeps quality moving forward and reduces risk associated with product
changes.
A test lead might ask these questions of someone automating
a test: In how many different ways might this automated test fail, and what
happens when it does fail?
Covey's character ethic, applied to software quality, makes
for stronger quality and stronger product character.
Here is a related post on the need for test code quality: http://metaautomation.blogspot.com/2011/09/if-product-quality-is-important-test.html
