Tuesday, January 5, 2016
The current paradigm of functional software quality is in crisis.
“Test automation” is broken. It’s not that it doesn’t bring some value – it does! – but we in the field all know of the many test automation projects that have failed to deliver. The current paradigm is in crisis. It’s time to look again.
The following is extracted from my draft of Chapter 2 of the 2nd Edition of MetaAutomation, due out in February or March. Interested engineers can also check out http://MetaAutomation.net, and the working samples there that illustrate the radical benefits of MetaAutomation.
Traditional model: automated tests are just like manual tests, but faster.
New paradigm: automated verifications are different from manual testing in almost every way.
Traditional model: Manual test cases, and automation of same, provide the model and limits for measuring quality.
New paradigm: When quality automation moves away from focusing on making the system under test do stuff and towards measuring and communicating quality, it can provide much greater business value than with current practices.
Traditional model: Checks generate logs, just like any other ongoing process.
New paradigm: Time-bound checks enable robust, structured artifacts with self-documenting steps, vastly increasing the value of the check artifacts.
Traditional model: Check steps are a linear list.
New paradigm: Check steps can be self-documenting and hierarchical, enabling business-facing steps as parents and technology-facing steps as children.
Traditional model: Automation of a product is limited to one tier for any given check.
New paradigm: Structured data allow any check to manipulate and measure across any number of tiers, enabling end-to-end checks for the Internet of Things.
Traditional model: Checks described with business-facing terms allow all team members to consume results, but require the QA role to maintain and hide the details of what’s going on.
New paradigm: With hierarchical self-documenting check steps, nothing is hidden and everything is exposed to any team member that wishes to drill down from business-facing parent steps to the child-most steps that describe atomic operations on the product.