The demand for higher-quality products and new features delivered at breakneck speeds has the average engineer struggling just to stay afloat. With demands rising but resources not always following suit, engineering teams are embracing automation as a key testing option.
According to a report by advisory firm KPMG, test automation increased by 85 percent from 2015 to 2017 across all industries. Automation is one way to address this accelerated development schedule, but it can also introduce other problems. To ensure that doesn’t happen, there are four common pitfalls that engineering teams need to look out for.
Problem No. 1: Right Job, Wrong Tools
It’s tough to build a shed with a spatula and a sewing needle. Likewise, you can’t build a successful test automation suite if the tools used don’t fit your company’s testing strategy.
The solution: Pick the right framework for your suite.
The first question you need to ask yourself is: Should you build or buy your test automation framework? The answer likely depends on how mature your testing structure is. Smaller teams new to the technology typically are best-served starting with an off-the-shelf enterprise solution. This approach makes it easier to get up and running quickly, while having the support to handle software bugs along the way.
Larger, more mature testing teams may have more to gain from an open source automation framework. This cost-effective approach provides flexibility and customization, but often requires advanced technical expertise and the luxury of time to build such a solution.
Problem No. 2: Living Fast, Dying Young
Many automation projects never make it off the ground because teams move too fast and automate too much, too soon. Inserting automation into the delivery pipeline prematurely will cause workflow and build issues more often than not.
The solution: Build test automation into your pipeline incrementally.
It may be tempting to run full speed as soon as you are set up, but the truth is you need to take your time for developers to build confidence in the testing process. Once that confidence is established and you’re receiving valuable feedback, you will have the full team’s support as you continue to scale your automation strategy.
Problem No. 3: Automation Without Documentation
Automation can make testing easy. Sometimes too easy, to the point where engineers will simply kick back a failed test without looking into the reason it failed in the first place. Oftentimes, these failures are the result of an outdated testing script or an issue with the test environment, and not with the code at all.
The solution: Triage automation test results just like manual bugs.
Instituting an intermediary triage stage will save your engineering team time in the long run. There’s no need to treat your processes for automation results any different than you would bugs found manually. Having access to materials such as activity logs and screenshots will only help to strengthen and speed the triage process.
Problem No. 4: Setting and Forgetting
Automation makes testing quicker and easier. That’s why you invested in it in the first place, right? Unfortunately, there is more to such solutions than just plugging them in and letting them run.
The solution: Take maintenance into account when building your framework.
Many teams are surprised by the maintenance required to keep test automation suites running smoothly and effectively. They shouldn’t be. Solutions need to be updated continually to address user interface changes, new operating systems and browsers, and automation library changes. The maintenance of third-party dependencies also needs to be considered before automation is used for production test runs.
Maintaining your automation suite is a continuous process. Make sure you have enough testing resources—either in-house, outsourced, crowdtesting or a combination of all three—to ensure your efforts don’t go to waste.
Measure Twice, Implement Once
Make no mistake, automation is more than a quick fix. If you are going to spend the time and money necessary to build a successful strategy, you should make sure to set yourself up with a solid foundation. A framework that is thrown together within days but falls apart in hours will only serve to make automation seem like an unpalatable solution within your testing department. Having the right partners in place and the right solution from the start will help automation become a reality in your company, and can bring the promises of speed and flexibility to life.
— Drew Horn
