DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • DevOps Chats
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Communities
    • AWS Community Hub
    • CloudBees
    • IT as Code
    • Rocket on DevOps.com
    • Traceable on DevOps.com
    • Quali on DevOps.com
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Digital Anarchist
  • Media Kit
  • About
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DevSecOps
  • DevOps Onramp
  • Practices
  • ROELBOB
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps

Home » Blogs » DevOps Practice » Quality at Speed During the COVID-19 Pandemic

Quality at speed

Quality at Speed During the COVID-19 Pandemic

By: Wolfgang Platz on June 4, 2020 1 Comment

Think back to 2015. The U.S. economy was in the early phases of a growth spurt that would continue for years. Ninety-five percent of organizations were practicing agile, to some extent (State of Agile, VersionOne). And the proportion of the IT budget allocated to QA and testing had just increased from 18% to 26% (World Quality Report, Capgemini). 

Recent Posts By Wolfgang Platz
  • Shift Left … With In-Sprint UI Test Automation (?!?)
  • Software Testing: The Comeback Kid of the 2020s
  • DevOps Places Software Testing in the Crosshairs
More from Wolfgang Platz
Related Posts
  • Quality at Speed During the COVID-19 Pandemic
  • Continuous Testing: Solving the Biggest DevOps Bottleneck
  • How Can DevOps Enable Your Software Testing efforts?
    Related Categories
  • Blogs
  • DevOps Practice
    Related Topics
  • AI-driven testing
  • linear expansion
  • progression testing
  • quality at speed
  • regression testing
  • software testing
  • stateful test data management
  • test automation
  • UI test automation
  • UI Testing
Show more
Show less

“Quality at speed” became the catchphrase of the year in the testing world. With all the buzz around DevOps and agile, it was clear that speed mattered. In reality, most teams weren’t quite there yet. Agile adoption was still low (less than 50% of teams within each organization actually practiced it) and maturity was even lower (only 17% of those practicing it reported mature adoptions). Still, the writing on the wall was clear. Leaders wanted more application development, faster — so testers should at least prepare for the pending storm.  

From that point forward, organizations across almost every industry all progressed to a sweet spot where time-to-market pressure was relatively high and cost pressure was moderate. By the end of 2019, quality at speed had graduated from buzzword to business as usual at most organizations. 

But then came COVID-19. Now, we’re in the midst of a global pandemic. Every business, every non-profit and every governmental agency is impacted — but in different ways. 

Speed expectations suddenly shifted into overdrive for an unlikely assortment of organizations, including government agencies and non-profits, healthcare and pharmaceutical companies, online retailers, online conferencing providers and telecoms. Yes, everyone had already been working to accelerate their software delivery processes. And many enterprise organizations had made great strides toward scaling the adoption of agile and DevOps. But, very few of these organizations were previously delivering applications at the extreme speeds required right now.

For another group, cost reduction suddenly became the top priority. Airlines, hotels, as well as other travel and leisure organizations experienced such dramatic changes in fortune that speed became a non-issue. The focus shifted to increasing cost efficiency in an effort to offset some of these staggering losses. 

Assuming that high quality cannot be compromised, this leaves both of these groups with distinctly different — yet equally daunting — challenges from a quality perspective. 

The organizations in the first group suddenly needed to build, test and release applications at the speed of a startup — but with the added burden of legacy systems, complex application landscapes and intense regulatory scrutiny. Quality at speed remains the goal, but the “speed” part is much, much faster. 

The organizations in the second group need to determine how to continue meeting strict quality expectations with a significantly slimmer testing budget. Quality at speed isn’t so important right now. Speed gets replaced by the other corner of the classic cost-quality-speed triangle, making the new mantra “quality at cost.” 

It’s hard to find good news in the current situation. But, from the software testing perspective at least, there’s some consolation. Whether you’re facing unprecedented time-to-market pressure or extreme cost pressure, the same core “test efficiency” strategies will help you either way. 

Here’s a quick overview of ways you can test more efficiently at different steps of the software testing process.

Selecting and Designing Test Cases

I’m addressing these two steps together because they both face a common challenge: intuition. Most modern business systems are insanely complex — so complex that it’s just not feasible for most testers to intuitively identify the best set of tests to cover a specific test need. Having a more systematic way to find the smallest — yet most powerful — set of tests will save you both time and money. 

For designing test cases for progression testing, methodologies like linear expansion and pairwise testing can help, and augmented intelligence can dramatically simplify them for you. 

For selecting what test cases should be run for regression testing, consider change impact analysis. Some approaches identify which objects are impacted by a given change, then help you narrow the test scope to the impacted objects. This often reduces the testing scope by 15%. Not bad, but you can do much better.

AI-powered change impact analysis pinpoints exactly what needs to be tested in a given update, based on whether the changes put an object at risk. With this approach, you can reduce testing efforts by 85% or more. Not only is this approach faster and cheaper, it’s much more efficient at uncovering defects. The number of defects that hit production are reduced by 75% to 95%, sometimes even 100% (no defects in production).

Automation and Execution

Here, technical scripting is the top time and cost drain. Some level of maintenance is required to keep scripts running, and this maintenance will cut into your test efficiency. One way to combat this challenge is to take a model-based approach to test automation. Such approaches have been field-tested for years, and proven to solve the test maintenance nightmare that’s doomed all-too-many test automation initiatives. 

The new kid on the block is AI-driven UI test automation. This is a big topic that warrants a much deeper discussion — but here’s a quick overview. Specify tests in natural language. Build the tests even before the actual UI is ready. Use the same tests as the technical implementation changes. Understand dynamic behavior like a human tester would. Test absolutely any technology — faster than a human eye. Curious? Stay tuned. 

Analysis of Results

When it comes to results analysis, false positives are the vampire sucking your resources. False positives are tests that fail even though the application is behaving correctly.  They will really kill you in terms of cost and speed. First, someone needs to figure out why the test failed. This might involve manually checking the related application functionality, inspecting the test automation, reviewing the test data, checking whether every element of the test environment is up and working as expected — maybe even all of the above. Once the exact problem is found, it needs to be fixed. Then, the test needs to be rerun. If it still fails, rinse and repeat.

How do you stop false positives? To start, stabilize the automation with model-based test automation or the AI-driven testing I mentioned in the previous section. As you move into more advanced automation scenarios, stateful test data management and service virtualization can help you stabilize tests to the point that they’re fit for CI/CD. Then, your final challenge is to deal with the flaky tests resulting from things like asynchronous application behavior, concurrency issues and test order dependencies. Again, AI and ML can offer tremendous assistance here—in flagging potential false positives, and even in helping you identify the cause and optimal resolution. 

Final Thoughts

COVID-19 split up the original quality at speed sweet spot into the extreme speed camp and the extreme cost-savings camp. In both cases, the same core strategies to increase your test efficiency will help you increase speed as well as reduce testing costs. If you’re under extreme time pressure, showing that you also reduced costs while meeting speed expectations certainly won’t hurt. If you’re currently reducing costs because your industry is essentially paused, taking these steps now will prepare you to speed it back up when the world becomes more normal again.

Filed Under: Blogs, DevOps Practice Tagged With: AI-driven testing, linear expansion, progression testing, quality at speed, regression testing, software testing, stateful test data management, test automation, UI test automation, UI Testing

Sponsored Content
Featured eBook
The State of the CI/CD/ARA Market: Convergence

The State of the CI/CD/ARA Market: Convergence

The entire CI/CD/ARA market has been in flux almost since its inception. No sooner did we find a solution to a given problem than a better idea came along. The level of change has been intensified by increasing use, which has driven changes to underlying tools. Changes in infrastructure, such ... Read More
« AppOmni Supports Remote Workers with Release of Enterprise Essentials for SaaS
Working with SAFe and Microservices: Learnings from Developers »

TechStrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

10 steps to continuous performance testing in DevOps
Thursday, August 11, 2022 - 3:00 pm EDT
Bring Your Mission-Critical Data to Your Cloud Apps and Analytics
Tuesday, August 16, 2022 - 11:00 am EDT
Mistakes You Are Probably Making in Kubernetes
Tuesday, August 16, 2022 - 1:00 pm EDT

Latest from DevOps.com

CloudNativeDay: WASM to Drive Next IT Epoch
August 10, 2022 | Mike Vizard
MLOps Vs. DevOps: What’s the Difference?
August 10, 2022 | Gilad David Maayan
GitHub Brings 2FA to JavaScript Package Manager
August 9, 2022 | Mike Vizard
CREST Defines Quality Verification Standard for AppSec Testing
August 9, 2022 | Mike Vizard
IBM Unveils Simulation Tool for Attacking SCM Platforms
August 9, 2022 | Mike Vizard

Get The Top Stories of the Week

  • View DevOps.com Privacy Policy
  • This field is for validation purposes and should be left unchanged.

Download Free eBook

The Automated Enterprise
The Automated Enterprise

Most Read on DevOps.com

Recession! DevOps Hiring Freeze | Data Centers Suck (Power) ...
August 4, 2022 | Richi Jennings
Orgs Struggle to Get App Modernization Right
August 4, 2022 | Mike Vizard
GitHub Adds Tools to Simplify Management of Software Develop...
August 4, 2022 | Mike Vizard
The Everything-As-Code Revolution and the OWASP Top 10
August 4, 2022 | Aakash Shah
Putting the Security Into DevSecOps
August 5, 2022 | Ross Moore

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2022 ·Techstrong Group, Inc.All rights reserved.