DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • Calendar View
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • Calendar View
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Cloud Native Now
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • CI/CD
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Sustainability
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • Chronosphere Adds Professional Services to Jumpstart Observability
  • Friend or Foe? ChatGPT's Impact on Open Source Software
  • VMware Streamlines IT Management via Cloud Foundation Update
  • Revolutionizing the Nine Pillars of DevOps With AI-Engineered Tools
  • No, Dev Jobs Aren’t Dead: AI Means ‘Everyone’s a Programmer’? ¦ Interesting Intel VPUs

Home » Blogs » DevOps Practice » ‘DevOoops’ Moves: Poorly Defined KPIs

‘DevOoops’ Moves: Poorly Defined KPIs

Avatar photoBy: Brian Dawson on February 22, 2019 1 Comment

DevOps’ ultimate aim is to create efficiencies in the software delivery process. But, it doesn’t always work out that way. Sometimes DevOps initiatives lead to mistakes—something we lovingly call a “DevOoops.” CloudBees recently polled some colleagues and came up with five examples of a DevOoops in action. The following is a discussion submitted by product marketer Juni Mukherjee about poorly defined key performance indicators (KPIs).

Recent Posts By Brian Dawson
  • SDM and the DevOps Delusion of Connected Organizations
  • Why DevOps is Like Football
  • ‘DevOoops’ Moves: Deploying Code Out of Step
Avatar photo More from Brian Dawson
Related Posts
  • ‘DevOoops’ Moves: Poorly Defined KPIs
  • ‘DevOoops’ Moves: SCM Conflicts
  • ‘DevOoops’ Moves: Deploying Code Out of Step
    Related Categories
  • Blogs
  • Continuous Delivery
  • DevOps Culture
  • DevOps Practice
    Related Topics
  • cloudbees
  • devops
  • key performance indicators
  • KPIs
  • testing
Show more
Show less

Scenario:

How we define KPIs can change the game. For example, one quality engineering team defined the number of tests executed per sprint as a success metric. Humans are driven by incentives and the natural tendency of the team was to add more and more tests without even considering archival of outdated ones. Think of this: We could have a larger impact with fewer tests and, more importantly, fewer tests lower the test cycle time. So, instead of sheer quantity, we should focus on the coverage and effectiveness of tests. Similarly, a release engineering team defined the number of releases per sprint as a success metric. The number of releases reflects on velocity for sure. However, releases move bits from point A to point B without assessing the value added to business. It is critical to tie a business KPI (such as the number of new customers acquired, percentage of revenue increase, etc.) to velocity, such that we know we are investing in valuable speed and not suicidal speed.

Management consultant Peter Drucker boiled his whole philosophy down to the following few words: “If you can’t measure it, you can’t improve it.”

Successful DevOps leaders take this saying to heart. To understand where they’re going, how to get there and ultimately how far they’ve come, these leaders know how critical it is to establish KPIs that fit the overall objectives of their DevOps transformations. KPIs provide appropriate and continuous insight—and they plot a route to help you get to your destination.

The big question is: Are you measuring the right things? If not, you could be spinning their wheels.

To avoid this “DevOoops” scenario, you need to start by establishing KPIs that not only align to your objectives but can also be established so that they create a continuous feedback loop. As the project progresses, as you mature and learn more, you will likely need to maintain, monitor and possibly adjust and update these KPIs.

What would be an appropriate test coverage goal? Teams all want to improve quality. But by how much? Seventy percent? Eighty percent? One hundred percent? This is an example of a KPI that needs to be considered in the context of your application as well as measured against your real objective.

You can have 70 percent test or code coverage, but if that’s just treated as a measure to test as much code as possible, you may miss the mark. If you are covering 70 percent of the wrong code and the 30 percent that’s missing is critical code, you’ve put in significant effort with the risk of little or no return on quality level.

In terms of your goals, increasing and monitoring test coverage may be an indicator of quality, but it may not measure your actual quality of work. In this case you want a KPI that’s measuring the outcome rather than the activity. Look at what is severity 1 and severity 2 defect rate in production. You can see code coverage get to 90 percent, but if you’re not seeing a decrease in severity 2 defects, you won’t really be seeing an improvement in our code coverage metric.

Takeaways: 

  • You’re not implementing DevOps just to say you’re doing it; you need to develop goals and objectives for your initiative and align to them.
  • Once goals are defined, you need to align KPIs and success metrics to those objectives. Properly defining these success metrics and KPIs will dictate what gets prioritized when building pipelines.

— Brian Dawson

Filed Under: Blogs, Continuous Delivery, DevOps Culture, DevOps Practice Tagged With: cloudbees, devops, key performance indicators, KPIs, testing

« Smashing the (DevOps) Stack for Fun and Profit
Root Access »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Upcoming Webinars

Securing Your Software Supply Chain with JFrog and AWS
Tuesday, June 6, 2023 - 1:00 pm EDT
Maximize IT Operations Observability with IBM i Within Splunk
Wednesday, June 7, 2023 - 1:00 pm EDT
Secure Your Container Workloads in Build-Time with Snyk and AWS
Wednesday, June 7, 2023 - 3:00 pm EDT

GET THE TOP STORIES OF THE WEEK

Sponsored Content

PlatformCon 2023: This Year’s Hottest Platform Engineering Event

May 30, 2023 | Karolina Junčytė

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Why a Low-Code Platform Should Have Pro-Code Capabilities

March 24, 2021 | Andrew Manby

AWS Well-Architected Framework Elevates Agility

December 17, 2020 | JT Giri

Latest from DevOps.com

Chronosphere Adds Professional Services to Jumpstart Observability
June 2, 2023 | Mike Vizard
Friend or Foe? ChatGPT’s Impact on Open Source Software
June 2, 2023 | Javier Perez
VMware Streamlines IT Management via Cloud Foundation Update
June 2, 2023 | Mike Vizard
Revolutionizing the Nine Pillars of DevOps With AI-Engineered Tools
June 2, 2023 | Marc Hornbeek
No, Dev Jobs Aren’t Dead: AI Means ‘Everyone’s a Programmer’? ¦ Interesting Intel VPUs
June 1, 2023 | Richi Jennings

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

Most Read on DevOps.com

What Is a Cloud Operations Engineer?
May 30, 2023 | Gilad David Maayan
Forget Change, Embrace Stability
May 31, 2023 | Don Macvittie
No, Dev Jobs Aren’t Dead: AI Means ‘Everyone’s a Programmer’? ¦ Interesting Intel VPUs
June 1, 2023 | Richi Jennings
Five Great DevOps Job Opportunities
May 30, 2023 | Mike Vizard
Checkmarx Brings Generative AI to SAST and IaC Security Tools
May 31, 2023 | Mike Vizard
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.