“We want to do continuous deployment” – i’ve heard this many times, but have yet to find a scenario where it actually fit. Why? Because you are confused. And rightly so. How many two letter acronyms can be thrown at you in one movement. DevOps takes the confusion further and has two of the acronyms identical, but meaning very different things. The CDs, which are not the round shiny discs in this case. This straight forward post hopefully helps clear up some of the confusion.
The beloved two letter acronyms are:
- Continuous Integration (CI)
- Continuous Delivery (CD)
- Continuous Deployment (CD)
And this is how they fit into the software delivery chain:
This is one of many such diagrams, but the quick visual is useful for the rest of the conversations. What is not represented here well is the feedback loop that is fluid and never ending between all stages back to coding. Also note that this is the exploded view, so it shows all elements. But depending on the environment, everything could be collapsed into Continuous Deployment. The difference being the manual check points.
Now the details:
Continuous Integration (CI)
Continuous integration is a concept introduced by Martin Fowler in Extreme Programming development methodology, as one of its twelve practices. The evolved version of CI supposes the existence of four elements :
- A testing strategy: Very important
- “Flexible” infrastructure. Containers, VMs that can be spun up and down on a whim by developers and QA without the need for IT.
- A version control tool like Git, CVS, SVN etc. with webhooks
- An automated build and software release process and tool like Jenkins, GO
- A feedback mechanism: if a single test fails, the team is alerted the “build is broken” and they can fix the problems and reach a stable release, as soon as possible
- A process for incorporating the source code changes, which can be manual or automatic (using a continuous integration server)
- Functional testing process and tools. This is often neglected, but in the best set up 75% of the entire process.
The advantages using CI
- “Fail Fast” do high risk testing with no risk to production
- Minimizes risks, by identifying code errors quickly
- Because of automation reduces repetitive manual processes
- Generates fast deployable software
- Enables better project visibility to all Devops team members
- Increases developer confidence in the software product, as feedback is provided early (also the earlier assessment is provided, the less will cost to fix)
Sounds like the right direction to go, so what are the concerns that prevents teams from adopting CI?
- Increased budget needed for maintaining the CI system, which is a wrong point of view as integration, testing, and deployment will be done anyway, and it’s not dependent on using or not CI
- Too much change, which can be dealt with using an incremental approach: first builds and tests with a lower occurrence need to be added (like daily builds), and afterwards increase the frequency, as everyone gets more pleased with the results
- Too many failed builds, which happens mainly when developers are not performing a private build before committing their code to the version control repository
- Additional hardware/software costs. True, but they well worth the extra expense, for example a separate integration machine should be purchased, which will compensate, later in the development lifecycle, the more costly process of finding problems. And the cloud solutions such as Sauce Labs, BrowserStack, Test Droid, etc to offload functional testing is a huge effort and time savings.
How is CI implementation done?
Start by selecting a CI environment ( cloud or on-prem) and take a number of factors into consideration in doing so:
- Do you want it on-premise or to be hosted? Your selection will depend on wanting to run the infrastructure yourself or wanting to invest into a hosting solution managed by a 3rd party service provider.
- What is the platforms you’re building for? If the hosted CI service is on Linux it may be suitable for Java, Ruby or Node.js, but not .NET, for example AppHarbor is.
- Scalability. As CI environments grow over time, they will require more hardware and more licensed software, so increased costs (a lot of CI packages are sold per build agent, so with an upper limit number of builds that can run concurrently). And again the cloud solutions are low risk, and huge help here. They can run tests in parallel with brand new environments each time.
CI servers are usually compact and easy to setup, use, and configure, which is integrated with a wide variety of distributed version control systems, testing frameworks and offers out of the box CI for any Ant, Maven or Gradle project and Jenkins, which is free and open source, widely-used and well documented, used in combination with a large number of tools and technologies and which has a massive community that supports it.
Once a CI server is selected a source control system should be considered, which will influence the build setup. Source Safe, Subversion, or Vault are more for development on a single branch with a related build, while Git or Mercurial provide invaluable support for development of features on multiple branches.
CI opened the door for other trends that are that are gaining momentum and sometimes are mistaken for CI: continuous delivery and continuous deployment.
Continuous Delivery (CD)
Continuous Delivery is about developing in a way that the software is always ready to be delivered to a production-like/stage environment and will run correctly.
Even if developers begin writing releasable code that satisfies all functional tests from day one of the project, it does not demand when to release software, just that whenever the release occurs, the code is ready for the users.
The idea is to make a continuous-delivery a push-button system, which results in many daily deployments.
The infrastructure is seen as code and handled as an application changes to the infrastructure, are immediately integrated and tested, and, if it passes, deployed, as the ones for software applications, in an iterative delivery process.
The goal is to make visible the progress of builds from check-in to delivery to everyone. This will allow to see which changes have broken the application and which produced release candidates good enough for manual testing or release.
In terms of configuration management, there are four elements that are important to continuous delivery:
- Ability to build, test and deploy the application in a fully automated way
- Consistent management of the application’s deploy-time and runtime configuration
- Possibility to create or apply a configuration change to every environment using a fully automated process.
- All development done on a mainline, with larger features and restructuring implemented incrementally, so that the application is always kept working and ability to allow incomplete features to be made unavailable using configuration settings in the UI, if needed.
- In case your organization it’s excited about continuous delivery implementation, it should be aware that attempting to build it over a non-existent or Unstable CI “layer” is a sure recipe for failure.
Continuous Deployment (CD)
Continuous deployment follows continuous delivery and deploys automatically all changes that passed the automated tests to production.
The main benefits for continuous deployment appear as a result of reducing lead time (the time elapsed between the identification of a requirement and its fulfillment), with two main consequences:
- For each feature, after it is developed, an earlier return on investment, which lessen the need for large capital investments
- Earlier evaluations from users on each new feature, which allows for A/B testing to establish which of several possible implementation is preferred by clients
Some of the fringe benefits are Growth Hacking and an extremely fast feedback loop from application usage to product management.
Continuous deployment has additional costs, as it relies on instrumentation, to ensure that new functionality does not result in bugs and also on infrastructure that allows effortlessly backing out new features when a defect has not been caught by automated tests.
Deciding to release a particular version of the application should be done just by pushing a button, with the certainty that the release being deployed has gone through the entire pipeline successfully.
CI and the two CDs can provide an outstanding environment for the entire DevOps team, while the benefits greatly outweigh the challenges, and high quality can be achieved even for very large capacity and scale systems, while delivering value and innovation very fast in same time.
If you put all the terms together it sounds a disease , I suffer from “CICD”.
At the end of the day CI is for everyone Delivery is for most, and Deployment is for the rare few. Deployment requires massive applications and rapid feedback loops that are only for the high user, and transaction volume applications.
Oh and by the way, everything I said above is invalid as I type it. The variations are as many as there are organizations doing it. And the approaches are always changing. But the important thing to realize is DevOps, and these concepts are a framework. And the desired results of CI, CD, CD never change.