The Delivery Pipeline is your DevOps Signature

Where is DevOps? Searching for it is like querying a rainbow for pots of gold. No single element in a development team’s lifecycle defines them as a DevOps-enabled organization. Were anything the rainbow and the gold, it would be the delivery pipeline. The deployment pipeline centers the team and the process, functioning as the signature of any DevOps shop.

Every organization has a deployment pipeline. They may not call it that, but to get their application from concept to production, they all work through steps that become sequential and eventually routine.

The deployment pipeline has changed a lot over the years, and has been managed by several methodologies, from waterfall, to agile and Kanban. When DevOps morphed into what it is today, the pipeline folded in on itself and the end point was indistinguishable from the start.

Though every organization has a pipeline, many don’t define it. Consequently, it’s hard to hold comprehensive conversations about the development processes. They tend to be specific around one component like QA, or unit testing. Rather than the entire process and team. This results in disjointed teams and processes, and this, my friends, is not DevOps.

The benefit of defining a pipeline is:

  1. Knowing what you have
  2. Knowing what you don’t have
  3. Knowing your bottlenecks
  4. Seeing your whole operation in one place
  5. And finally having an opportunity to improve

I’ve spoken to several dev shops that were not born bottom up as DevOps organizations. Their operations have been evolving alongside their applications, and the results have been mixed.

Some made the move successfully and others did not. Some did so all at once and others took some time. I discovered that they all struggled with where to begin. Many would jump into some tool, only to find that the splash damage disrupted everything in the operation.

Some decided to start over with a brand new team and run two operations in parallel for a while. This approach worked wonders, but the political, and human resource disruption lasted for some time.
With those who were willing to evolve over time, the unanimous answer of where to start was to document the pipeline and look for places to automate and improve. This approach allows them to build in DevOps one section at a time, preparing the team’s functions and culture to fail fast and iterate fast. This also allows the larger organization to evolve over time.

I can’t point to the exact right approach. While the last two have sustained so far, the first is still the most common. And while the idea in DevOps is to move quickly, there has to be some meta-level plan for it to be successful. Tools are great but they are not a magic bullet.

Pipeline components are often very similar:

1.) Plan

  • a. Roadmap
  • b. Tickets
  • c. Backlog

2.) Code
3.) Build
4.) Test

  • a. Unit Test
  • b. Integration Test
  • c. Functional Test

5.) Release

  • a. To Staging
  • b. To Production

6.) Operate

  • a. Monitor
  • b. Report

7.) Rinse and Repeat

Some organizations might skip steps, but the categories are ubiquitous. The details populating each category are not. Every organization has a different team, tool set, and processes within each category. There is no cookie cutter pipeline, because applications are not cookie cutter. If they were, there would be no need for developers.

It is these details that many neglect in their pipeline documentation. For example, how do you do integration testing? Who does it? What tools do you use? What is automated and what is not? What can you automate? Conversations about tools happen casually every day across groups and subjects, but rarely are their discussions about the comprehensive unit. Neglecting to do so allows non-deliberate changes to the processes resulting in FrankensteinOps.

For each stage in the pipeline I expect to see:

Who does it?
How do they do it?
How it’s working well?
How it could be improved?


Each time you complete this exercise, you have a single snapshot of your entire process. And as you keep updating this pipeline view, you will see how the team has improved. For example, if you introduce an automated component to a manual process – maybe Jenkins for automated release to Staging – then check that your manual release to production still works, you have taken one major step forward without serious disruption.

Creating, documenting, and building conversations around your delivery pipeline are the best way to transition teams to DevOps. This will slowly but surely enhance your development processes.

About the author  ⁄ Chris Riley

Chris Riley

Chris Riley (@HoardingInfo) is a technologist and DevOps analyst for Fixate IO. Helping organizations make the transition from traditional development practices to a modern set of culture, tooling, and processes that increase the release frequency and quality of software. He is an O'Reilly author, speaker, and subject matter expert in the area of DevOps Strategy, Machine Learning, and Information Management.