DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • HPE to Acquire OpsRamp to Gain AIOps Platform
  • Oracle Makes Java 20 Platform Generally Available
  • How to Maximize Telemetry Data Value With Observability Pipelines
  • Awareness of Software Supply Chain Security Issues Improves
  • Why Observability is Important for Development Teams

Home » Features » Meeting Challenges in incorporating automated testing throughout the DevOps cycle

Meeting Challenges in incorporating automated testing throughout the DevOps cycle

Avatar photoBy: David Geer on November 11, 2015 Leave a Comment

Enterprise development teams find it difficult to perform QA tests on software in short DevOps cycles where they cannot squeeze in lengthy regression tests. DevOps teams must insert automated testing approaches to resolve this predicament. “DevOps teams need to be more selective in prioritizing tests that occur at each stage of the integration to delivery pipeline,” says Marc Hornbeek, Senior Solutions Architect of DevOps, Spirent.

Recent Posts By David Geer
  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 2
  • Q&A: BDO’s Coffman on Change Management, Security and DevOps, Part 1
  • Sounding the Death Knell for Agile: Not so Fast!
Avatar photo More from David Geer
Related Posts
  • Meeting Challenges in incorporating automated testing throughout the DevOps cycle
  • Rugged DevOps: From Idea to Automation to Impact
  • Performance Testing: The Unexpected Missing Link in DevOps
    Related Categories
  • Features
    Related Topics
  • Automated testing
  • QA
Show more
Show less

At each point between stages of the development lifecycle, there is an opportunity to insert the most applicable tests to determine whether the software has incurred errors. It is crucial to test early. The earlier in the development process that team members discover their errors, the sooner they can fix them, ensuring fewer error-related issues in releases. The earlier in the development lifecycle that errors occur and go unchecked, the more opportunity they have to grow and become increasingly debilitating to the software.

With best practices, DevOps shops can use the right tests in the right stages of development and achieve quality assurance without depending on large test suites that run at the end of the cycle, according to Hornbeek.

The best time to create automated tests is together with or prior to developing the code for the given software feature itself. This will enable development shops to keep pace with the speed of DevOps; the old approach of automating testing after the feature is developed has truly become ineffective, according to Hornbeek.

DevOps Targets Quality / Requires Testing

DevOps has the goal of creating better software faster. Apart from effective, continuous, automated testing at keenly selected intervals, DevOps is only faster. This would achieve only a form of dev similar to what we had before DevOps, i.e., the kind where coders believe they have done well even though operations staff sees a lack of fitness of the software for production use.

Enterprises must embed a culture of QA inside the culture of DevOps rather than separate out QA testing from DevOps using organizational silos; communicating only through inefficient bug reporting methods is no longer the state-of-the-art, let alone practical with the increasing industry expectations of DevOps.

Testing: Why Must I Be Misunderstood?

The impact of testing is often overlooked. DevOps shops that take testing for granted may not invest in or improve upon testing as testing needs change and grow. “This was true before DevOps and continues to be true with DevOps. Developers focus on feature creation, while the business leaders focus on delivery,” says Hornbeek. As a result, testing is often not a focus of DevOps.

Further, there is a common misconception that testing just the part that is done after the builds are created is all that is necessary; but enterprises must integrate continuous automated testing with DevOps throughout the development-to-delivery pipeline, according to Hornbeek.

“When testing is conducted a long time after the feature is developed, there are many inefficiencies. The developers lose context of the problem and have a harder time recreating, diagnosing, and fixing problems,” says Hornbeek. Automated continuous testing eliminates this delay, improves software, and shrinks time to quality software releases.

Slothful Testing Gone Awry

There are high-profile examples of slothful software testing where projects grew out of scope and way off target. A chief example is the original rollout of Healthcare.gov, says Hornbeek. “In another example, a network equipment manufacturer tried to implement DevOps twice without success. They finally realized that they had underestimated continuous testing,” says Hornbeek of a Spirent customer.

To prevent these kinds of outcomes, says Hornbeek, follow these best practices:

• Create automated tests as you develop the features.
• Do not accept features into the integration branch without automated tests.
• Ensure that the developer test environment is a good representation of the production environment.
• Select DevOps-ready tools that have Rest APIs.
• Test priority features so you can easily integrate them into the DevOps orchestration framework.

Follow these steps, according to Hornbeek, to incorporate automated testing for QA:

• Establish a clear leader with responsibility over the entire dev-to-deliver pipeline who is visible and will champion continuous testing.
• Assign an architect for continuous testing tools selection and to oversee the technical implementations.
• Determine best practices for Continuous Testing and decide what practices best fit the organization.
• Address the culture, roles, and responsibilities for continuous testing.
• Put some automated metrics in place, such as a goal to automate 85% of all tests.

Editors Note: I had a chance to catch up with Mark Hornbeek at the Enterprise Summit 2015. Here is my interview with Mark.

Filed Under: Features Tagged With: Automated testing, QA

« A 6-pack for building a business case for CD
APM on-premises or SaaS: What’s the best fit? »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

The Testing Diaries: Confessions of an Application Tester
Wednesday, March 22, 2023 - 11:00 am EDT
The Importance of Adopting Modern AppSec Practices
Wednesday, March 22, 2023 - 1:00 pm EDT
Cache Reserve: Eliminating the Creeping Costs of Egress Fees
Thursday, March 23, 2023 - 1:00 pm EDT

Sponsored Content

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Practical Approaches to Long-Term Cloud-Native Security

December 5, 2019 | Chris Tozzi

Latest from DevOps.com

HPE to Acquire OpsRamp to Gain AIOps Platform
March 21, 2023 | Mike Vizard
Oracle Makes Java 20 Platform Generally Available
March 21, 2023 | Mike Vizard
How to Maximize Telemetry Data Value With Observability Pipelines
March 21, 2023 | Tucker Callaway
Awareness of Software Supply Chain Security Issues Improves
March 21, 2023 | Mike Vizard
Why Observability is Important for Development Teams
March 21, 2023 | John Bristowe

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

GET THE TOP STORIES OF THE WEEK

Most Read on DevOps.com

Large Organizations Are Embracing AIOps
March 16, 2023 | Mike Vizard
Modern DevOps is a Chance to Make Security Part of the Process
March 15, 2023 | Don Macvittie
Addressing Software Supply Chain Security
March 15, 2023 | Tomislav Pericin
What NetOps Teams Should Know Before Starting Automation Journeys
March 16, 2023 | Yousuf Khan
DevOps Adoption in Salesforce Environments is Advancing
March 16, 2023 | Mike Vizard
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.