When we look beyond the fundamentals of tearing down silos and establishing greater collaboration, communication and transparency, DevOps is a label assigned to a framework of best practices and technologies that organizations leverage to deliver systems faster, more efficiently and without sacrificing quality. The best practice within that framework is continuous improvement, which provides stakeholders with data and feedback to constantly improve their ability to deliver products that advance strategic business objectives. This is done through key performance indicators (KPIs) that align with the goals of the business.

Continuous improvement thrives on a robust program of KPIs that measure key attributes of a successful development team in the context of what is important to your organization. These KPIs provide you with the opportunity to continuously evaluate your system delivery processes and supporting technologies, and identify opportunities to make changes that produce better outcomes. In general, all organizations should be continuously improving their system development and delivery against three primary KPIs:

• Quality: The value and performance of software you deliver to customers.
• Velocity: The speed at which you develop and deliver software to customers.
• Efficiency: The ease (low resource consumption, minimal latency) with which you develop and deliver software to customers.

For now, let’s focus on what’s required to continuously measure and improve against them.

Continuous Improvement KPIs vs. Metrics

KPIs provide baselines for minimally acceptable performance levels across critical areas that impact the success of your business. Metrics measured against these KPIs help development teams improve what they’re doing to support the performance of the business in light of its goals.

There are hundreds of metrics available (escaped versus trapped defects, features deployed and mean time to X, to name a few). The problem is most organizations focus too much on non-business-related metrics and data points and end up measuring the wrong things, such as how many lines of code have been written. This data may be interesting, but it tells you nothing of how well or within what time frame the code was written, or how that affects the delivery of value to customers.

Your metrics should align with the goal-oriented KPIs of your business. For example, if a goal is to increase the quality of software after an influx of customer support issues related to defects, you should measure how many defects are escaping and how many are being trapped in testing. However, the remaining challenge for most organizations is actually capturing this metrics data to accurately measure progress and continuous improvement against, for example, a quality baseline.

Collecting Data for Continuous Improvement

A large reason organizations struggle to capture the right metrics data is they rely on outdated project management tools and manual effort. Developers and other team members painstakingly capture and contribute metrics data through status reports and meetings. Project managers then spend the bulk of their time analyzing and producing even more reports to provide the necessary feedback to support continuous improvement.

In a digital economy where fast beats slow, this slow approach of manual data collection constrains and negates the business objectives of increasing quality, velocity and efficiency simultaneously. Core to Agile is the production of working software and minimizing the amount of work not directly related to that endeavor. If you want to move to an agile, iterative world, you want to minimize activity or any work that’s not directly related to the software delivery effort.

That’s why, for true continuous improvement, you want your tools to automatically produce the right metrics data to measure and continuously improve upon KPIs, rather than relying on developers to attend status meetings and feed project managers information used to manually build reports.

Once you’re able to visualize this data with a modern dashboard-driven analytics tool, you can use that information to constantly assess and evaluate system development and delivery productivity, and help developers make behavioral changes. You will have the necessary information to tweak processes and tooling and better educate team members.

For example, if your software quality drops below its KPI-defined baseline and your metrics show increased efforts utilized to resolve break/fix issues, you can make a valuable correlation leading to smarter actions to improve quality, such as tweaking or changing testing methods and possibly using more effective testing tools.

In this way, continuous improvement becomes embedded in your software delivery architecture, as you’re no longer making gut-based decisions that require you to wait for an outcome and guess again. Instead, you’re using data to drive strategic, fact-based shifts in behavior as you develop software, with the knowledge that these changes will have a real impact on the business.

Making Continuous Improvement Across Platforms

Ultimately, continuous improvement must reach across platforms in your organization—from the cloud to the mainframe. Development teams should work to continuously improve system delivery across all development platforms.

The KPIs mentioned must apply to the entire development team and business, not just developers on one particular platform or skill set. You want to identify KPIs that are critical to the successful delivery of systems that support your customers, employees and business partners.

Now that we have covered the importance of continuous improvement through the use of system delivery KPIs, stay tuned as we explore the individual topics of quality, velocity and efficiency at a deeper level in subsequent articles.

— Rick Slade