Post #3 of 6: Make your check fails self-documenting

If MetaAutomation is too much change for right now…

Verify that you’re doing all you can to minimize debugging costs, in case of check failure.

1.       Add preliminary verifications wherever null-reference exceptions occur

A null-reference exception (or null-pointer exception for a native language) will lose information: which symbol was null? There is often more than one possibility on a given line number. If there’s an ambiguity, it might require a debug session to find out, and in some cases, the line number might have changed for the version of the code you can access versus the one that was running and threw the exception.

Labeled preliminary verifications, e.g., asserting non-null “<some API> returned null>” will find the condition sooner and show it with much better specificity than if there were no such verification. With complex code this can make the difference between a clearly understood root cause of check failure, and a very confused one.

If there isn’t a preliminary verification for where the null could potentially happen, and the check failure can’t be reproduced easily, a debug session can be either very expensive or even useless, meaning that the information of the failure is lost forever.

2.       Verify that asserts or verifications are self-documenting

Labels for thrown exceptions add value in two ways: First, they describe what happened, to make resolution of a check failure easier, and second, they ensure that there is no ambiguity of where or under what circumstances the failure occurred. Using the built-in exceptions can make sense for produce code where the standard for performance is high, but for quality automation code, use an assert or a custom exception to make the failure explicitly self-describing.

3.       Add more log statements around points of failure

Consider the difference in cost between adding a few more log statements around parts of checks that are likely to fail, and reproducing and debugging through the failure, maybe more than once, especially if the failure is hard to reproduce.

Adding log statements is much cheaper and lower-risk.

4.       Make all checks fail on an exception, even negative cases

For negative checks that are expected to fail, enclose the expected failure in code that verifies the expected failure condition, then throw another exception in case of failure to fail the check.

Exceptions are intended to be used in exceptional circumstances. Overuse of exceptions, or too much code that is expected to be bypassed in case of an exception, reduces reusable code in the quality automation infrastructure because a new pattern would be required for coding the negative check method signatures, and makes any cleanup more complex.

The Atomic Check pattern of MetaAutomation shows how to do all this with much more structure, transparency, and detailed hierarchical self-documenting check code.

This page is #3 in a series of six, with the original post here.