Post #2 of 6: Basic Check Design

If MetaAutomation is too much change for right now…

Here are some easy steps to take:

1.       Make your checks independent of each other.

This is how checks can be fast and scalable: they must be independent of each other. Even the simplicity of the checks depends on them being independent of each other, and in remember in case of a failure, the value of what a check can find can depend on simplicity, because with long or complex checks, things can get muddy and any business value can be clouded and even lost because it becomes too much work to debug through the check to try to get the actionable information out, surface it and describe it for the benefit of the larger team.

This is a core principle of the Atomic Check pattern of MetaAutomation: all checks are independent of each other. This is how checks can scale across resources. If the “Chained Tests” antipattern is still used, checks can’t scale at all because they still have to happen in a time sequence, and any failure might depend on an operation earlier in the sequence, so the value of any issues found depends first on a lot of work to reproduce that sequence of operations.

2.       Prioritize your checks based on business requirements (or even functional requirements)

It seems obvious, doesn’t it? Prioritize… work on the more important parts of your product first.

If your team doesn’t have business requirements, you probably don’t know what you’re building, and the risk is that you’re building the wrong thing for your customers. See the excellent book by Robin Goldsmith on this topic, for example.

At least, however you’re defining your requirements (e.g., with user stories), prioritize them and prioritize your checks based on requirements.

This is part of the Atomic Check pattern as well.

3.       Ensure that your automated checks are completely repeatable. If needed, record the parameters used so you can repeat them as needed.

Whether or not you’ve ever had a failure in any one check, you need to be able to repeat the check exactly.

Otherwise, you can’t be sure that quality is always getting better because what aspect of quality you measured before isn’t necessarily the aspect you’re measuring now, and maybe you just got lucky today.

In case of failure, you need a record to do the failure again, or if you can’t reproduce it, go looking for what the real problem is, e.g., a race condition somewhere.

This is part of the Atomic Check pattern as well.

4.       Move setup and teardown operations out-of-line.

Any check might have preliminary setup steps and/or tear-down steps that can be moved out-of-line. The question is this: can any setup or teardown be done asynchronously and in some other process, in some other memory space?

For example, allocating and initializing an environment in which to run a check; that should be done out-of-line.

Allocating, initializing or re-initializing a user identity: that should be done out-of-line.

Any other state for the check, e.g. some file image or system state, that can be done out-of-line and out-of-process, should be done that way, because it means that your checks can run faster and more scalably.

The Precondition Pool pattern of MetaAutomation addresses this.

This page is #2 in a series of six, with the original post here.