The current paradigm of functional software quality is in
crisis.
“Test automation” is broken. It’s not that it doesn’t bring
some value – it does! – but we in the field all know of the many test
automation projects that have failed to deliver. The current paradigm is in
crisis. It’s time to look again.
The following is extracted from my draft of Chapter 2 of the
2nd Edition of MetaAutomation, due out in February or March. Interested
engineers can also check out http://MetaAutomation.net,
and the working samples there that illustrate the radical benefits of
MetaAutomation.
Paradigm Shift 1: Automated Verifications, aka Checks, Are Very Different from the Manual Test Role
Traditional model: automated tests are just like manual
tests, but faster.
New paradigm: automated verifications are different from
manual testing in almost every way.
Paradigm Shift 2: Focusing Checks On Their Unique, Important Capabilities Vastly Increases Business Value
Traditional model: Manual test cases, and automation of same,
provide the model and limits for measuring quality.
New paradigm: When quality automation moves away from
focusing on making the system under test do stuff and towards measuring and
communicating quality, it can provide much greater business value than with
current practices.
Paradigm Shift 3: Checks are Bounded in Time, Enabling Structured Artifacts
Traditional model: Checks generate logs, just like any other
ongoing process.
New paradigm: Time-bound checks enable robust, structured
artifacts with self-documenting steps, vastly increasing the value of the check
artifacts.
Paradigm Shift 4: Hierarchical Steps Give Transparency, Robustness, and Flexibility
Traditional model: Check steps are a linear list.
New paradigm: Check steps can be self-documenting and
hierarchical, enabling business-facing steps as parents and technology-facing
steps as children.
Paradigm Shift 5: The Internet of Things is Distributed, So Measure Quality That Way Too
Traditional model: Automation of a product is limited to one
tier for any given check.
New paradigm: Structured data allow any check to manipulate
and measure across any number of tiers, enabling end-to-end checks for the
Internet of Things.
Paradigm Shift 6: Better Quality Data leads to Quality Power Across the Team and Across the SDLC
Traditional model: Checks described with business-facing
terms allow all team members to consume results, but require the QA role to
maintain and hide the details of what’s going on.
New paradigm: With hierarchical self-documenting check
steps, nothing is hidden and everything is exposed to any team member that
wishes to drill down from business-facing parent steps to the child-most steps
that describe atomic operations on the product.
