DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • DevOps Chats
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Communities
    • AWS Community Hub
    • CloudBees
    • IT as Code
    • Rocket on DevOps.com
    • Traceable on DevOps.com
    • Quali on DevOps.com
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Digital Anarchist
  • Media Kit
  • About
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DevSecOps
  • Leadership Suite
  • Practices
  • ROELBOB
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps

Home » Features » Meeting Challenges in incorporating automated testing throughout the DevOps cycle

Meeting Challenges in incorporating automated testing throughout the DevOps cycle

By: David Geer on November 11, 2015 Leave a Comment

Enterprise development teams find it difficult to perform QA tests on software in short DevOps cycles where they cannot squeeze in lengthy regression tests. DevOps teams must insert automated testing approaches to resolve this predicament. “DevOps teams need to be more selective in prioritizing tests that occur at each stage of the integration to delivery pipeline,” says Marc Hornbeek, Senior Solutions Architect of DevOps, Spirent.

Recent Posts By David Geer
  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 2
  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 1
  • Sounding the Death Knell for Agile: Not so Fast!
More from David Geer
Related Posts
  • Meeting Challenges in incorporating automated testing throughout the DevOps cycle
  • How DevOps is Killing QA
  • This is Not Just a Test: DevOps and the Need to Automate
    Related Categories
  • Features
    Related Topics
  • Automated testing
  • QA
Show more
Show less

At each point between stages of the development lifecycle, there is an opportunity to insert the most applicable tests to determine whether the software has incurred errors. It is crucial to test early. The earlier in the development process that team members discover their errors, the sooner they can fix them, ensuring fewer error-related issues in releases. The earlier in the development lifecycle that errors occur and go unchecked, the more opportunity they have to grow and become increasingly debilitating to the software.

DevOps Connect:DevSecOps @ RSAC 2022

With best practices, DevOps shops can use the right tests in the right stages of development and achieve quality assurance without depending on large test suites that run at the end of the cycle, according to Hornbeek.

The best time to create automated tests is together with or prior to developing the code for the given software feature itself. This will enable development shops to keep pace with the speed of DevOps; the old approach of automating testing after the feature is developed has truly become ineffective, according to Hornbeek.

DevOps Targets Quality / Requires Testing

DevOps has the goal of creating better software faster. Apart from effective, continuous, automated testing at keenly selected intervals, DevOps is only faster. This would achieve only a form of dev similar to what we had before DevOps, i.e., the kind where coders believe they have done well even though operations staff sees a lack of fitness of the software for production use.

Enterprises must embed a culture of QA inside the culture of DevOps rather than separate out QA testing from DevOps using organizational silos; communicating only through inefficient bug reporting methods is no longer the state-of-the-art, let alone practical with the increasing industry expectations of DevOps.

Testing: Why Must I Be Misunderstood?

The impact of testing is often overlooked. DevOps shops that take testing for granted may not invest in or improve upon testing as testing needs change and grow. “This was true before DevOps and continues to be true with DevOps. Developers focus on feature creation, while the business leaders focus on delivery,” says Hornbeek. As a result, testing is often not a focus of DevOps.

Further, there is a common misconception that testing just the part that is done after the builds are created is all that is necessary; but enterprises must integrate continuous automated testing with DevOps throughout the development-to-delivery pipeline, according to Hornbeek.

“When testing is conducted a long time after the feature is developed, there are many inefficiencies. The developers lose context of the problem and have a harder time recreating, diagnosing, and fixing problems,” says Hornbeek. Automated continuous testing eliminates this delay, improves software, and shrinks time to quality software releases.

Slothful Testing Gone Awry

There are high-profile examples of slothful software testing where projects grew out of scope and way off target. A chief example is the original rollout of Healthcare.gov, says Hornbeek. “In another example, a network equipment manufacturer tried to implement DevOps twice without success. They finally realized that they had underestimated continuous testing,” says Hornbeek of a Spirent customer.

To prevent these kinds of outcomes, says Hornbeek, follow these best practices:

• Create automated tests as you develop the features.
• Do not accept features into the integration branch without automated tests.
• Ensure that the developer test environment is a good representation of the production environment.
• Select DevOps-ready tools that have Rest APIs.
• Test priority features so you can easily integrate them into the DevOps orchestration framework.

Follow these steps, according to Hornbeek, to incorporate automated testing for QA:

• Establish a clear leader with responsibility over the entire dev-to-deliver pipeline who is visible and will champion continuous testing.
• Assign an architect for continuous testing tools selection and to oversee the technical implementations.
• Determine best practices for Continuous Testing and decide what practices best fit the organization.
• Address the culture, roles, and responsibilities for continuous testing.
• Put some automated metrics in place, such as a goal to automate 85% of all tests.

Editors Note: I had a chance to catch up with Mark Hornbeek at the Enterprise Summit 2015. Here is my interview with Mark.

Filed Under: Features Tagged With: Automated testing, QA

Sponsored Content
Featured eBook
DevOps: Mastering the Human Element

DevOps: Mastering the Human Element

While building constructive culture, engaging workers individually and helping staff avoid burnout have always been organizationally demanding, they are intensified by the continuous, always-on notion of DevOps.  When we think of work burnout, we often think of grueling workloads and deadline pressures. But it also has to do with mismatched ... Read More
« A 6-pack for building a business case for CD
APM on-premises or SaaS: What’s the best fit? »

TechStrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Deploying Microservices With Pulumi & AWS Lambda
Tuesday, June 28, 2022 - 3:00 pm EDT
Boost Your Java/JavaScript Skills With a Multi-Experience Platform
Wednesday, June 29, 2022 - 3:30 pm EDT
Closing the Gap: Reducing Enterprise AppSec Risks Without Disrupting Deadlines
Thursday, June 30, 2022 - 11:00 am EDT

Latest from DevOps.com

Developer’s Guide to Web Application Security
June 24, 2022 | Anas Baig
Cloudflare Outage Outrage | Yet More FAA 5G Stupidity
June 23, 2022 | Richi Jennings
The Age of Software Supply Chain Disruption
June 23, 2022 | Bill Doerrfeld
Four Steps to Avoiding a Cloud Cost Incident
June 22, 2022 | Asim Razzaq
At Some Point, We’ve Shifted Too Far Left
June 22, 2022 | Don Macvittie

Get The Top Stories of the Week

  • View DevOps.com Privacy Policy
  • This field is for validation purposes and should be left unchanged.

Download Free eBook

The State of the CI/CD/ARA Market: Convergence
https://library.devops.com/the-state-of-the-ci/cd/ara-market

Most Read on DevOps.com

Survey Uncovers Depth of Open Source Software Insecurity
June 21, 2022 | Mike Vizard
One Year Out: What Biden’s EO Means for Software Devs
June 20, 2022 | Tim Mackey
Open Source Coder Tool Helps Devs Build Cloud Spaces
June 20, 2022 | Mike Vizard
Not Everything That is Necessary Adds Value
June 20, 2022 | Lance Knight
TechStrong Con: Downturn Brings Additional Sense of DevOps U...
June 21, 2022 | Mike Vizard

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2022 ·Techstrong Group, Inc.All rights reserved.