Patterns and antipatterns in automation (part 2 of 3-4 parts)

For a product with significant impact, a pattern is writing test automation in the same language as the product (for Java) or the same framework as the product (for .Net). An antipattern is scripting some other, lighter-weight language.

I know many perceive that a script language e.g. Python, Perl or Javascript is better suited to write test automation because it may be quicker. But, with a strongly-typed compiled language, the compiler is your friend, finding all sorts of errors early. With a decent IDE, intellisense is your friend as well, hinting at what you can do and backing that up with the compiler.

If test automation is written in a different language than the product, then testers are distanced from the product code enough that they don’t have anything to say about it; it’s not their domain. But, product code is usually a good example to follow when writing good, robust, extensible test code. Today at work I entered two significant bugs in product code that I wouldn’t have been able to do if I weren’t continually up with the languages used in the product (a grammar of XML, and C#).

Another reason for having the test code in the same language (or framework) as the product is that you know that no information is lost with remote calls or exceptions rolling up the stack: the stack is the same through product and test code, and the test code can handle product exceptions if that’s called for in the test.

The barrier to entry with C# is actually quite low; a dev can help, and testers don’t need to get fancy with lambda expressions, delegates and partial class declarations. If a tester were to create robust, extensible, OO code anyway, a powerful language is needed.

I’ve seen many problems with interfacing a script language or GUI test framework with a compiled OO language: awkwardness of error handling or verification, dropped information, limited design choices…

What do you think? Do you have a favorite language or framework for test automation, that’s different than the product language?

Topic to be continued tomorrow …