DevOps.com

  • Latest
    • Articles
    • Features
    • Most Read
    • News
    • News Releases
  • Topics
    • AI
    • Continuous Delivery
    • Continuous Testing
    • Cloud
    • Culture
    • DataOps
    • DevSecOps
    • Enterprise DevOps
    • Leadership Suite
    • DevOps Practice
    • ROELBOB
    • DevOps Toolbox
    • IT as Code
  • Videos/Podcasts
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
    • DevOps Unbound
  • Webinars
    • Upcoming
    • Calendar View
    • On-Demand Webinars
  • Library
  • Events
    • Upcoming Events
    • Calendar View
    • On-Demand Events
  • Sponsored Content
  • Related Sites
    • Techstrong Group
    • Cloud Native Now
    • Security Boulevard
    • Techstrong Research
    • DevOps Chat
    • DevOps Dozen
    • DevOps TV
    • Techstrong TV
    • Techstrong.tv Podcast
    • Techstrong.tv - Twitch
  • Media Kit
  • About
  • Sponsor
  • AI
  • Cloud
  • CI/CD
  • Continuous Testing
  • DataOps
  • DevSecOps
  • DevOps Onramp
  • Platform Engineering
  • Sustainability
  • Low-Code/No-Code
  • IT as Code
  • More
    • Application Performance Management/Monitoring
    • Culture
    • Enterprise DevOps
    • ROELBOB
Hot Topics
  • There’s No Value in Observability Bloat. Let’s Focus on the Essentials
  • Observability Leaders: The Mighty Minority Holding Their Own Against Outages
  • Cisco Acquires Splunk to Create Observability Powerhouse
  • Nobl9 Unfurls Reliability Center for Managing SLOs
  • Harness Launches Open Source Gitness Platform

Should You Measure Developer Productivity?

Should You Measure Developer Productivity?

Avatar photoBy: Bill Doerrfeld on September 18, 2023

A recent McKinsey article entitled ‘Yes, you can measure software developer productivity‘ has got the software developer community talking. And the general consensus is they’re not happy.

Most software developers are not enthusiastic about the prospect of having their individual performance metrics monitored. “Anyone who attempts to measure software in some sort of scientific way is just doing the wrong thing,” said YouTube personality ThePrimeTime. Dave Farley of Continuous Delivery even goes as far as to call the McKinsey report “harmful dribble.”

This debate arises as developer productivity engineering (DPE) is gaining traction as a means to improve developer performances and, perhaps, in response to employment landscape shifts like the Great Resignation, which saw developers and DevOps pros leaving their roles. Companies like Spotify, LinkedIn, Netflix and others are also investing in internal developer platforms (IDPs) to increase developer productivity and satisfaction amid a limited talent supply.

AWS Builder Community Hub

Alongside these ambitions, much has been written about how managers can measure developer productivity in terms of qualitative metrics, like developer satisfaction, as well as quantitative metrics, like number of commits or story points, deployment frequency or failure rates. But can tracking such metrics deliver tangible results? Or do they go too far?

Below, we’ll consider whether you can actually measure software developer productivity. But more than that, we’ll evaluate whether this is a meaningful tactic with desirable outcomes.

Ways to Measure Development Productivity: DORA

First, let’s look at some of the standard techniques to measure development productivity. The DORA metrics, produced by the DevOps Research and Assessment (DORA) team at Google after years of research, identified four key metrics:

  1. Deployment frequency: How often you ship changes.
  2. Lead time for changes: The time between a commit and it reaching production.
  3. Change failure rate: The percent of releases that fail; downtime, worsened service or rollbacks.
  4. Time to restore service: The average time it takes to respond to an issue, also known as mean-time-to-recovery.

These metrics could be helpful indicators of a software development team’s overall agility and health. They can signal bottlenecks in the CI/CD process or highlight where too much time is spent remediating issues, encouraging positive changes. Solving these root causes could all improve a team’s productivity.

DevOps World 2023

Going Beyond DORA: SPACE

DORA also added reliability to the list. That’s understandable, given the interest in building reliable systems. But things start to get a little more complex when SPACE enters the equation.

Proposed in a 2021 paper by GitHub researchers, SPACE augments DORA with the following attributes:

  • Satisfaction and well-being
  • Performance
  • Activity
  • Communication and collaboration
  • Efficiency and flow

Metrics like satisfaction or communication and collaboration are pretty subjective. Plus, it can be challenging to gather this data and encourage engineers to speak honestly in surveys. Measuring the activity of individual members, like the number of issued closed or pull requests made, isn’t exactly tied to end value and could even produce a negative psychological effect.

“Consequently, teams can become fixated on them, as seemingly positive changes are relatively easy to achieve,” wrote James Walker. He adds that SPACE still doesn’t tell you why measurements have value—furthermore, it lacks actionable information and might be tricky to implement, given developer pushback against performance analysis systems.

Opportunity-Focused Metrics

The McKinsey report adds to the mix what it calls “opportunity-focused productivity metrics,” which include inner/outer loop time spent, a Developer Velocity Index benchmark, contribution analysis and a talent capability score. (It’s not totally clear how these metrics can be computed). It then organizes all these metrics into a framework that further segments measurements into system, team and individual levels.

McKinsey's framework for measuring developer productivity

While some believe in the framework’s legitimacy, many commentators don’t see the practicality of implementing such a framework in practice. “Introducing the kind of framework that McKinsey is proposing is wrong-headed and certain to backfire,” wrote the PragmaticEngineer.com. “The report is so absurd and naive that it makes no sense to critique it in detail,” wrote Ken Beck.

Things to Avoid With Developer Productivity Metrics

Apparently, developers are very weary of being measured. And given the recent backlash, it seems like overreliance on productivity metrics could negatively impact the engineering culture. So, engineering leadership should be careful when implementing these sorts of tactics. Here are some anti-patterns to consider.

Don’t make it about individual performances. Metrics like DORA have become standards for measuring the efficiency of software teams, but they shouldn’t be applied at the individual level. “Measuring software development in terms of individual developer productivity is a terrible idea,” said Dave Farley. Technology development is more of a collaborative process; performance metrics should reflect how the team operates.

Don’t measure developer productivity in isolation. On that note, performance metrics shouldn’t be judged in isolation. Issues with failures, for example, often depend on contexts outside of an individual developer’s control. At the same time, it can be hard to correlate individual developer contributions to end customer satisfaction.

Don’t create an opportunity to gamify productivity systems. Leadership shouldn’t directly tie a developer’s salary, reputation or position to whether or not they hit certain metrics. If productivity systems carry this much weight, they could become gamefied.

Don’t put in more effort than it’s worth. In tech, where there’s hype, there’s money to be made—and a host of vendors ready to hawk monitoring tools. What’s more, to enact these productivity measurement enhancements, McKinsey noted that “most companies’ tech stacks will require potentially extensive reconfiguration.” Devoting many engineering resources to monitor engineers just seems like a big pill to swallow.

Don’t put too much faith in metrics. Lastly, metrics only show one side of the picture. They are only data points and can’t always depict what it’s really like on the ground. Furthermore, qualitative responses also might be skewed due to reluctance to speak up.

Remember Goodhart’s Law and Focus On Quality

Personally, I believe that measuring things like developer productivity, efficiency, experience and satisfaction does have some merit for some businesses. Most teams are now working asynchronously, and metrics help leaders take the pulse of a team’s performance, even if they are all remote workers.

It’s also nothing new. Leadership has been attempting to measure developer productivity for years. But, these metrics often lack context and come with many caveats to consider. Thus, they shouldn’t be given a huge seat at the table.

I like this ‘law’ from British economist Charles Goodhart, which is colloquially summarized as “when a measure becomes a target, it ceases to be a good measure.” In other words, when you start working for the metrics, the engineering culture and overall code quality might decline. This matters, since a separate study found that increases in code quality tend to be followed by an increase in developer productivity.

So, focus first on establishing code quality and, hopefully, productivity will flow.

Recent Posts By Bill Doerrfeld
  • Innersourcing Open Source Principles in the Enterprise
  • Tips for Increasing Developer Velocity in the AI Age
  • Internal Developer Platform (IDP) Reference Architectures
Avatar photo More from Bill Doerrfeld
Related Posts
  • Should You Measure Developer Productivity?
  • A Measure of Developer Productivity
  • How LinkedIn is Investing in Developer Productivity and Satisfaction
    Related Categories
  • Application Performance Management/Monitoring
  • Blogs
  • Business of DevOps
  • DevOps Culture
  • DevOps Practice
  • Features
  • News
    Related Topics
  • developer productivity
  • devX
  • employee engagement
  • McKinsey
Show more
Show less

Filed Under: Application Performance Management/Monitoring, Blogs, Business of DevOps, DevOps Culture, DevOps Practice, Features, News Tagged With: developer productivity, devX, employee engagement, McKinsey

« Five Great DevOps Job Opportunities
Splunk: Creating a Resilient and Dynamic Organization »

Techstrong TV – Live

Click full-screen to enable volume control
Watch latest episodes and shows

Neo Quiz

Upcoming Webinars

Infosys Zero Cost Mainframe Transformations
Monday, September 25, 2023 - 11:00 am EDT
How PRINCE2 Improves Cybersecurity
Tuesday, September 26, 2023 - 11:00 am EDT
AWS and Sumo Logic: Observability With OpenTelemetry
Tuesday, September 26, 2023 - 1:00 pm EDT

GET THE TOP STORIES OF THE WEEK

Sponsored Content

JFrog’s swampUP 2023: Ready for Next 

September 1, 2023 | Natan Solomon

DevOps World: Time to Bring the Community Together Again

August 8, 2023 | Saskia Sawyerr

PlatformCon 2023: This Year’s Hottest Platform Engineering Event

May 30, 2023 | Karolina Junčytė

The Google Cloud DevOps Awards: Apply Now!

January 10, 2023 | Brenna Washington

Codenotary Extends Dynamic SBOM Reach to Serverless Computing Platforms

December 9, 2022 | Mike Vizard

Latest from DevOps.com

There’s No Value in Observability Bloat. Let’s Focus on the Essentials
September 22, 2023 | Tomer Levy
Observability Leaders: The Mighty Minority Holding Their Own Against Outages
September 22, 2023 | Jeff Stewart
Cisco Acquires Splunk to Create Observability Powerhouse
September 21, 2023 | Mike Vizard
Nobl9 Unfurls Reliability Center for Managing SLOs
September 21, 2023 | Mike Vizard
Harness Launches Open Source Gitness Platform
September 21, 2023 | Mike Vizard

TSTV Podcast

On-Demand Webinars

DevOps.com Webinar ReplaysDevOps.com Webinar Replays

Most Read on DevOps.com

Should You Measure Developer Productivity?
September 18, 2023 | Bill Doerrfeld
JFrog swampUP: Addressing the Advent of AI
September 18, 2023 | William Willis
Buildkite Acquires Packagecloud to Streamline DevOps Workflows
September 19, 2023 | Mike Vizard
What DevOps Teams Should Know About Phishing and the Supply Chain
September 19, 2023 | Gilad David Maayan
Oracle Previews Latest Java 21 Innovations
September 19, 2023 | Mike Vizard
  • Home
  • About DevOps.com
  • Meet our Authors
  • Write for DevOps.com
  • Media Kit
  • Sponsor Info
  • Copyright
  • TOS
  • Privacy Policy

Powered by Techstrong Group, Inc.

© 2023 ·Techstrong Group, Inc.All rights reserved.