Innovative MetaAutomation

Is MetaAutomation actually innovative?

From the Wikipedia entry on “Innovation:”

Innovation is a new idea, more effective device or process. Innovation can be viewed as the application of better solutions that meet new requirements, inarticulated needs, or existing market needs. This is accomplished through more effective products, processes, services, technologies, or ideas that are readily available to markets, governments and society. The term innovation can be defined as something original and more effective and, as a consequence, new, that "breaks into" the market or society.

By this definition, yes, MetaAutomation is very innovative. Here are some ways that MetaAutomation speaks to the Wikipedia entry:

New Idea

What happens after or as a result of an automation run is now explicitly important, with details on why this is true and what can be achieved with strong, focused data on the automation run. It goes far beyond what can be achieved with conventional automation or even automation that leaves no more than a flight-recorder log and a Boolean result as artifacts.

You could even call that the “Meta” of MetaAutomation.

MetaAutomation includes several paradigm shifts:

1.       Manual testing and automated testing bring different value to the team, and if done effectively, they will excel in different ways. If automation is focused on what it does well, it has quality value far beyond what the team gets by automating manual test cases.

2.       For automation artifacts, a strongly-typed, focused, pure data solution is much better than a flight-recorder log stream: more compact, richer information, presentation can be applied later so it is flexible (this is especially easy with XML and XSL, and optionally XSD as well, all of them W3C standards), and much more powerful in downstream solutions e.g. analysis, flexible reporting and automated communications.

Ever tried to automate parsing of streamed flight-recorder logs, where the format of the log entries includes English grammar and there are few format rules? I have. It’s nearly impossible to do this reliably.

Even given established practices, current investments, and the dissonance of introducing ideas that are very new and different, MetaAutomation (or, something a lot like it) is inevitable for software that matters to people.

More Effective Process

MetaAutomation includes the process or processes by which quality information is communicated and even made actionable and interactive around the team. This really only works when automation is reliable, fast and scalable, clear and actionable.

Without MetaAutomation, there tends to be failed automation that is ignored because it is assumed, usually correctly, that automation failure is not about a product quality issue. If the automation has recently failed, resolution to an action item often must wait until one of the automation authors has time to reproduce the problem and step through to debug the issue, which again is usually nothing to do with actual product quality.

Inarticulated Needs

To my knowledge, the need to have automation be reliable, actionable, self-documenting and interactive throughout the team has not ever been articulated as part of one concept (although, the reliable part has been attempted before at least, and the interactive part exists, depending on team process). Quality information that covers all business-logic behaviors and persists data for efficient and robust analysis over any time span represented by the data? That might exist for specific proprietary applications such as aviation or military, but it’s not publicly available, and in any case, I suspect that MetaAutomation does it better.

My last blog post addressed the connection between MetaAutomation and Sarbanes-Oxley (SOX) compliance, and in turn, company valuation (for a publicly-traded software company). This is a real value, but AFAIK it has not been articulated before.

Readily Available

MetaAutomation is a pattern language, a series of five design patterns with a defined dependency. It’s language-independent, although the freely-available sample implementation is written in C#, as is the library and sample that is nearly complete. The upcoming lib puts performance information into the XML elements that describe the steps of the check, and is distributed so can run a single check against multiple (virtual) machines.

The book defines the patterns with UML diagrams, clear reasoning, references etc. and describes the sample code.

Original

The closest analog I can find to MetaAutomation is the book “xUnit Design Patterns” by Meszaros. This book is useful, and I reference it from the MetaAutomation book, but there are major differences as well:

xUnit Design Patterns is a survey of existing practices. It does a nice job of describing the patterns and in some cases connecting them together, but the patterns are not original with Meszaros. It’s a large set of patterns, most of which I would not recommend following for specific engineering and quality reasons I won’t go into here.

Some of the patterns of MetaAutomation (e.g. Precondition Pool, Parallel Run) are related to existing patterns and practices in software engineering. Smart Retry is related to a Retry pattern in test automation practice, but MetaAutomation is able to do it much more efficiently because it depends on Atomic Check, the least dependent and possibly most original pattern of MetaAutomation.

The pattern language is very original, and it’s necessary to show the dependency. With Atomic Check, you can achieve a strong return in the value of your quality measurements, but also, if you choose, you and your team can go on to implement the other patterns of MetaAutomation.

Breaks

MetaAutomation is not incremental, nor additive. It’s not a tool you can buy.

I’m not saying that existing automation practices don’t add value; in most cases, they do.

I am saying that the value of MetaAutomation can only be achieved by approaching automation differently from conventional practices. The adoption bar is high because MetaAutomation might require you to start over with automation for software quality.

Innovation

For me, MetaAutomation has been a labor of love. I love software that works, and I love addressing big-picture issues with intensity, tenacity and intellectual courage. It hasn’t been easy, and sometimes my personality pushes people away, but I’m proud to think differently.