Analytics and MetaAutomation

Analytics are very powerful and an awesome developing technology, and your team needs automated verifications of business requirements as well (e.g. using the Atomic Check pattern of MetaAutomation, and dependent patterns).

Why both?

Atomic checks are fast and reliable enough to verify requirements from an end-user perspective quickly, and a set of them are even fast and scalable enough to gate check-ins (especially with the Precondition Pool, Parallel Run and Smart Retry patterns of MetaAutomation). Implementing these patterns can ensure that quality is always moving forward, which is especially important for software that matters.

Analytics can give the team ideas of what the end uses are experiencing. This is awesome, because otherwise you’d have little or no idea of what’s going on for your end users. This is very valuable for, e.g., A/B testing. OTOH, it’s somewhat like the shadows of Plato’s cave: you have to reconstruct and guess what’s really going on in some cases, e.g., through session abandonments on a site.

Analytics need product instrumentation, aka telemetry. Good news; there’s synergy here! Automated verifications can use those as well. Analytics benefits from product instrumentation through logs, and automated verifications can use the same outputs but synchronized for inclusion in the artifact of a check run. The valid XML document created by an atomic check is ideal for this.

 

This diagram gives a view of how product instrumentation can benefit both analytics and automated verifications (checks).