Wednesday, March 19, 2014

MetaAutomation: The Abstract for PNSQC (October, 2014)

Regression testing automation provides an important measure of product quality and can keep the quality moving forward during the SDLC. Unfortunately automation can take a long time to run, and automation failures generally must be debugged and triaged by the test automation team before any action item can be considered or communicated to the broader team. The resulting time lag and uncertainty greatly reduces the value of the automation, and causes cost, risk etc.

MetaAutomation is a language of five patterns that provide guidance to new and existing automation efforts, to provide fast and reliable regression of correct business behavior for a software solution and to speed quality communication around the team, while reducing latency and human cost.

The five patterns – Atomic Check, User Pool, Parallel Run, Smart Retry and Automated Triage - are in a sequence, representing an order in which the patterns can be applied, and also form a network of dependencies between the patterns.

For an existing automation project, the Atomic Check pattern can be applied in whole or in parts to run the automation faster (e.g. with shorter and better-defined tests) and create results which are more actionable (e.g. with asynchronous and/or inline test setup, hierarchical steps defined at runtime, explicit verifications, custom exceptions, etc.). If enough of Atomic Check is adhered to, the dependent patterns can then be applied to further speed, direct and enhance the value of communications resulting from the automation.

The patterns are language-independent. A platform-independent sample implementation of the Atomic Test pattern will be demonstrated in C#.