DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv Video Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Where Does Observability Stand Today, and Where is it Going Next?
  • Five Great DevOps Job Opportunities
  • A Freelancer's Workflow
  • Azure Migration Strategy: Tools, Costs and Best Practices
  • Blameless Integrates Incident Management Platform With Opsgenie

Home » Blogs » Continuous Testing » 4 Steps To Achieve a 66% Reduction in Test Run Time

4 Steps To Achieve a 66% Reduction in Test Run Time

Avatar photoBy: Nikolay Advolodkin on August 12, 2020 2 Comments

Here are ways organizations can reduce their test run times to improve speed and efficiency

Related Posts
  • 4 Steps To Achieve a 66% Reduction in Test Run Time
  • Continuous Testing Improves Speed, Quality
  • DevOps – It’s all about continuous testing !
    Related Categories
  • Blogs
  • Continuous Testing
  • DevOps Practice
    Related Topics
  • atomic testing
  • code quality
  • code testing
Show more
Show less

Consult any recent DevOps survey and the same theme emerges: Testing remains one of the biggest roadblocks impeding organizations’ collective efforts to deliver better software, faster. Unlike days past, however, when agile development was still in its relative infancy and investments in testing lagged other areas of the delivery pipeline, organizations today are increasingly aware of the critical role continuous testing plays in delivering quality software at speed, and are aggressively investing in the requisite testing tools and technologies.

TechStrong Con 2023Sponsorships Available

No, the problem today is not lack of awareness or tooling. The problem, quite simply, is that testing is difficult, and only gets more difficult as organizations look to drive automation at scale. Nowhere is this challenge more evident than in organizations’ ongoing struggles with test quality. A recent study showed only 24.37% of organizations pass at least 90% of their desktop tests and only 25.45% pass at least 90% of their mobile tests. Remember, the purpose of agile development is to accelerate application delivery and shorten the release cycle. Organizations can do neither if they constantly have to manually follow-up to determine the source of repeated test failures.

Doing It Right

Organizations must find ways to improve test quality and speed. A major reason so many tests fail is they take far too long to run. The longer it takes to run, the more likely it is to end with a failure. According to the aforementioned study, those that complete in two minutes or less are twice as likely to pass as those lasting longer than two minutes.

With that in mind, let’s take a look at four key steps testers and developers can implement to reduce test run times.

Script Atomic Tests

There’s no more important best practice to create a better and faster automation experience than to run atomic tests. These focus on just one piece of application functionality and are much faster and easier to execute than longer tests assessing multiple pieces of functionality.

Let’s examine what this looks like in practice. Suppose an online retailer needs to validate that users can log in, view their gift card balance, add items to their cart, proceed to checkout and successfully process a transaction. Even an experienced tester will often make the mistake of scripting a single test to validate those five functions. The far better approach, especially if the aim is to reduce run time and improve test quality, is to write five separate tests, each specifically focused on one piece of functionality. Running individually but in parallel, these five “atomic” tests will execute in far less time than one longer test.

Run Tests in Parallel

Of course, if those five atomic tests are executed sequentially—that is, one at a time, one after the other—they’ll take even longer to complete than the one screen flow test, negating any potential runtime benefit. That’s why running tests in parallel is an equally critical component of any strategy to reduce test run times. Consider the hypothetical example of a test suite with 100 atomic tests, each of which takes two minutes to execute. Run those 100 tests sequentially and they’ll take more than three hours to complete. Run them in parallel, and all 100 will complete in just two minutes.

Remember, too, not to get thrown by the volume of tests in a given suite. It’s tempting to think that a suite with 10 long tests that each take 5 minutes to complete will execute faster than one with 100 atomic tests that each take 1 minute to execute. But even if executed in parallel, the suite with the 10 long tests will still take 5 minutes to run—five times longer than the suite with 100 atomic tests.

Reduce the Number of Selenium Commands

Having too many Selenium commands in your test script flies in the face of atomic testing and is one of the most common underlying causes of long and unstable tests. Every single command takes time to execute and represents a new opportunity for something to go wrong. Minimize the number of commands required to execute a test case and run time will shrink accordingly.

Use Explicit Waits

Another effective way to reduce test run time is to use explicit waits rather than implicit waits. Implicit waits set a default wait time between each step or command across a test script, such that the subsequent step only executes after the pre-defined amount of time has elapsed. Explicit waits, on the other hand, enable the next step in a script to execute as soon as the preceding step is complete. Though more complicated to implement, using only explicit waits can have a significant positive impact on test run time.

Putting it Together

The example below shows an unoptimized test suite with 311 commands run against a mobile device. This example took 1,265 seconds (or more than 21 minutes) to execute. If delivering better software faster is the aim, that’s not going to work.

Applying the aforementioned tactics to the same test suite executed against the same device, however, drove the total number of commands down to just 127 and enabled the suite to execute in just 430 seconds (or just over 7 minutes), a 66% improvement in test run time.

Though the prospect of reducing test run time may be daunting, by focusing on a limited set of key tactics and best practices, the end goal of shorter, more stable tests is indeed attainable.

Filed Under: Blogs, Continuous Testing, DevOps Practice Tagged With: atomic testing, code quality, code testing

« Healthcare Increasingly Embracing DevOps
Upcoming Event: Operationalizing Kubernetes Virtual Summit »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Automating Day 2 Operations: Best Practices and Outcomes
Tuesday, February 7, 2023 - 3:00 pm EST
Shipping Applications Faster With Kubernetes: Myth or Reality?
Wednesday, February 8, 2023 - 1:00 pm EST
Why Current Approaches To "Shift-Left" Are A DevOps Antipattern
Thursday, February 9, 2023 - 1:00 pm EST

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

Where Does Observability Stand Today, and Where is it Going Next?
February 6, 2023 | Tomer Levy
Five Great DevOps Job Opportunities
February 6, 2023 | Mike Vizard
Azure Migration Strategy: Tools, Costs and Best Practices
February 3, 2023 | Gilad David Maayan
Blameless Integrates Incident Management Platform With Opsgenie
February 3, 2023 | Mike Vizard
OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot | Netflix Blocks Password Sharing
February 2, 2023 | Richi Jennings
Automation Challenges Holding DevOps Back
February 1, 2023 | Mike Vizard
Jellyfish Adds Tool to Visualize Software Development Workflows
January 31, 2023 | Mike Vizard
Cisco AppDynamics Survey Surfaces DevSecOps Challenges
January 31, 2023 | Mike Vizard
Red Hat Brings Ansible Automation to Google Cloud
February 2, 2023 | Mike Vizard
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.