DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • DevOps Chats
    • DevOps Unbound
  • Webinars
    • Upcoming
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • On-Demand Events
  • Sponsored Communities
    • AWS Community Hub
    • CloudBees
    • IT as Code
    • Rocket on DevOps.com
    • Traceable on DevOps.com
    • Quali on DevOps.com
  • Related Sites
    • Techstrong Group
    • Container Journal
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Digital Anarchist
  • Media Kit
  • About
  • AI
  • Cloud
  • Continuous Delivery
  • Continuous Testing
  • DevSecOps
  • Leadership Suite
  • Practices
  • ROELBOB
  • Low-Code/No-Code
  • IT as Code
  • More Topics
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps

Home » Blogs » Continuous Delivery » Clean Up Those Ever-Fail Tests

Clean Up Those Ever-Fail Tests

By: Don Macvittie on June 14, 2017 Leave a Comment

One of the points of DevOps is to streamline development and focus on what is important. Things that don’t raise to a high enough level of importance end up in the category of backlog, and their tests are normally disabled or marked as “ignore results” in CI/CD tools.

Recent Posts By Don Macvittie
  • Is Your Future in SaaS? Yes, Except …
  • Update Those Ops Tools, Too
  • Why We Still Need Specialists
More from Don Macvittie
Related Posts
  • Clean Up Those Ever-Fail Tests
  • Dagger: Standardizing CI/CD is the Holy Grail of DevOps
  • “I want to do Continuous Deployment”
    Related Categories
  • Blogs
  • Continuous Delivery
    Related Topics
  • agile development
  • continous testing
Show more
Show less

With CI/CD, this is a good step in moving forward, because you have a list of the things that you haven’t gotten to yet, even if that list is not currently part of validation. The problem is that it can create the very kind of cruft that we adopted CI/CD to get rid of. Many organizations keep these tests in the queue for an indefinite amount of time on the assumption that they will come in handy when priority allows for time to address the issues at hand.

DevOps/Cloud-Native Live! Boston

It Isn’t Simple

The problem is, of course, that some of those tests will never—and I do mean never—be resolved. Those are cruft, sitting in the way of productivity and cluttering up the build system. It can be a corporate decision that a given bug fix/feature request will never be implemented, but more often than not it is simply that the priority will never be addressed because there are always new features, more important bugs, etc.

Purists will already have started sweating while reading this, because the goal is supposed to be to address every single test in the system, resolve every single bug in the system, and implement every change request that made it far enough to have tests written for it. But we don’t live in a purist world. We never have. I’ve argued with purists of every possible stripe in my time in high-tech, because reality isn’t pure. It’s interruptive and ugly and prioritized.

So we have backlogs. And we have items in the backlog that won’t ever be addressed.

Why Old Tests Matter

Image of binThe real problem with tests that are backlogged for more than a few sprints is, of course, the changing code base. As an abstract example, you have a dialog that asks the user for some input. The tests assume the input values that are designed in at the time the test was written. Six months later, a higher-priority test comes through that changes what fields are available on the dialog, and what data is collected. The original test is marked as ignored, so when it fails that doesn’t impact things. But now the test is invalid. And still in your system. Multiply these two requests by hundreds or even thousands, and you have the same type of flotsam that DevOps was designed to resolve, except now it is part and parcel of the CI/CD system.

Clean Up, Acknowledge and Move On

So what is a productive team to do? Have a frank discussion about your backlog and what will/won’t ever be implemented. Business leaders can agree that some items just never will raise to the correct level of priority, and you can identify tests that have aged out of usefulness while sitting in the backlog.

Set up a process to review them occasionally, to run parallel to the process that updates tests when assumptions change—just not as frequently.

Then document the tests that won’t ever succeed, and delete them. They don’t belong in version control, even as documentation, because eventually and inevitably, they will be inaccurate representations of the system. And that can create confusion.

Of Course It’s DevOps

This is an extension of all that DevOps did to begin with: eliminate wasteful activity and non-productive items on the list of to-dos. You prioritized once and cut with a knife, this is the same process, operationalized for CI/CD. And it’s necessary.

A slight change, another set of choices that are no harder than the others you’ve faced, and the code base (in particular the test base) is cleaner.

Leaving you more time to keep rocking it.

— Don Macvittie

Filed Under: Blogs, Continuous Delivery Tagged With: agile development, continous testing

Sponsored Content
Featured eBook
The Automated Enterprise

The Automated Enterprise

“The Automated Enterprise” e-book shows the important role IT automation plays in business today. Optimize resources and speed development with Red Hat® management solutions, powered by Red Hat Ansible® Automation. IT automation helps your business better serve your customers, so you can be successful as you: Optimize resources by automating ... Read More
« Another Career Path: Full Stack Developer
Managing Complexity in Dev/Test Infrastructure Using Machine Learning »

TechStrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Accelerating Continuous Security With Value Stream Management
Monday, May 23, 2022 - 11:00 am EDT
The Complete Guide to Open Source Licenses 2022
Monday, May 23, 2022 - 3:00 pm EDT
Building a Successful Open Source Program Office
Tuesday, May 24, 2022 - 11:00 am EDT

Latest from DevOps.com

DevSecOps Deluge: Choosing the Right Tools
May 20, 2022 | Gary Robinson
Managing Hardcoded Secrets to Shrink Your Attack Surface 
May 20, 2022 | John Morton
DevOps Institute Releases Upskilling IT 2022 Report 
May 18, 2022 | Natan Solomon
Creating Automated GitHub Bots in Go
May 18, 2022 | Sebastian Spaink
Is Your Future in SaaS? Yes, Except …
May 18, 2022 | Don Macvittie

Get The Top Stories of the Week

  • View DevOps.com Privacy Policy
  • This field is for validation purposes and should be left unchanged.

Download Free eBook

The State of the CI/CD/ARA Market: Convergence
https://library.devops.com/the-state-of-the-ci/cd/ara-market

Most Read on DevOps.com

Why Over-Permissive CI/CD Pipelines are an Unnecessary Evil
May 16, 2022 | Vladi Sandler
Apple Allows 50% Fee Rise | @ElonMusk Fans: 70% Fake | Micro...
May 17, 2022 | Richi Jennings
DevOps Institute Releases Upskilling IT 2022 Report 
May 18, 2022 | Natan Solomon
Making DevOps Smoother
May 17, 2022 | Gaurav Belani
Creating Automated GitHub Bots in Go
May 18, 2022 | Sebastian Spaink

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2022 ·Techstrong Group, Inc.All rights reserved.