It can sometimes feel frustrating to try to figure out how to do DevOps. That’s because DevOps is not about tools—there are lots of different tools that can help you succeed at DevOps. Nor is it about any really specific set of practices; although DevOps prioritizes things like automation and communication, it does not prescribe specific processes for enabling them.
Instead, DevOps is really just about cultural values, which is a rather vague guideline for people who want to do DevOps. In this sense, DevOps can make practitioners feel a little lost, because they’re unsure how to measure whether they are succeeding on their journey toward DevOps.
The fact that DevOps ultimately boils down to culture does not mean there are not specific points of information you can track and analyze to help assess progress on your DevOps journey. The specific metrics you’ll analyze will vary depending on your circumstances and the systems you use, of course. But to provide an example of how you can evaluate DevOps progress in the real world, let’s take a look at which types of key performance indicators (KPIs) you might focus on when doing Z DevOps.
What Is Z DevOps?
Z DevOps refers to a DevOps practice within the IBM Z ecosystem. Although Z is not typically the first technological niche to come to most people’s minds when they think of DevOps, the fact is more than half of enterprise transactions continue to run on IBM mainframes. The vast majority of structured business data also resides on these systems.
Thus, bringing mainframe software delivery up-to-speed with DevOps culture is an important priority for many organizations. That’s what Z DevOps is all about.
KPIs for Z DevOps
Whether you’re just starting on your Z DevOps journey, or have been doing DevOps-based software delivery on the mainframe for a long time already, there are a variety of KPIs you can track to help assess how effectively your DevOps strategy is working, and find ways to make it even better. Remember, DevOps is all about continuous improvement and never settling for the current state of affairs.
Deployment Speed: DevOps prioritizes faster software delivery. For that reason, being able to track how quickly you push out new software releases or updates is critical for measuring your DevOps success. In DevOps circles, this KPI is sometimes called Mean Time to Delivery, or MTTD.
Keep in mind that truly continuous releases (meaning releases that are literally constant) are an unachievable goal. But by tracking your current deployments per week or month, you can measure how your deployment speed changes over time.
Deployment Success Rate: Software deployments can fail for a variety of reasons, and achieving 100% success is not practical for most organizations. But you still want to strive to maximize your deployment rate. Anything around 90% is impressive in many cases. So, track how many deployments succeed and fail.
Deployment Scale: Deployments vary greatly in size. One deployment might contain a few minor performance tweaks. Another might involve major interface or feature overhauls. In order to contextualize other deployment-related KPIs, it’s important to measure deployment scale by collecting information about the scale of your typical deployment.
Scale in this context can be difficult to measure precisely, but basic metrics to track include information like how much new code is a part of each deployment, or which percentage of a deployment entails new features as opposed to bug fixes for existing features.
Test Failure Rates: In most DevOps delivery pipelines, you will rely heavily on tests (especially automated tests) to verify that software is ready for delivery before it is deployed. When tests fail more often than expected, it can be a sign that developers are working too quickly or need more feedback about code quality. Tracking test failure rates can help you assess these issues and take action in order to minimize failures and keep deployments running on schedule.
Team Performance: In addition to tracking overall speed and performance, it’s important to be able to track these KPIs on a per-team basis. Again, DevOps is all about culture, and some teams might need more help than others in embracing DevOps culture. By tracking deployment performance on a per-team basis, you are in a better position to identify teams that need more help, as well as those doing well and can serve as an example for others.
Inter-Team Communication: Since DevOps is all about transparency and efficient communication across the entire IT organization, communication activity between different teams is another useful KPI to track. The data you collect on this front could include things such as emails or Slack messages exchanged between teams, or even meetings or phone calls. This data is not a perfect reflection of how effectively different teams engage with each other, but it provides some useful insight.
Collecting DevOps KPIs for Z
If you are doing Z DevOps, one challenge you’ll face is many generic DevOps tools weren’t initially designed to work in the Z ecosystem. They were designed for other types of infrastructure and environments.
That doesn’t mean the tools you need to do Z DevOps don’t exist, however. You just need to look beyond generic DevOps guides to find them.
When it comes to collecting KPIs for Z DevOps, the go-to tool is UrbanCode Velocity. In addition to helping to automate and standardize the software delivery process for the Z ecosystem by providing release automation templates and tooling, UrbanCode Velocity lets you collect and track the various metrics that are important for taking stock of your Z DevOps operation.
Because UrbanCode Velocity supports KPIs based on data gathered from Z DevOps pipelines, it can help to standardize your entire DevOps workflow, not just your Z DevOps effort.