As we move from continuous delivery to continuous experimentation, we also must develop new practices
As software development processes have evolved, we have mostly said goodbye to the idea of defined product versions. Many modern product delivery teams are taking this a step further—even the concept of a “product release” is starting to fade. Instead, products are becoming a fluid, rapidly evolving set of features, assembled uniquely for any given user.
Twenty-five years ago, the new release of a software product was a notable event. The marketing department would even insist on a catchy version name for the release. Internet Explorer 6, Office 2000, Windows XP, OSX Mountain Lion. This is still the case for a few types of software—operating systems being the most notable—but for most of the software products we use daily the concept of a version has faded away. What version of Google Maps are you using these days? How about Facebook? Twitter? As far as the end user is concerned, there is no version, just “current.”
A few different forces have moved software products away from release versions. The rise of agile software methodologies and associated practices such as continuous delivery has allowed software teams to tighten up software delivery cycles dramatically. Products that once were released every few months are now updated every few days.
In parallel, the growing capabilities of the web as a platform has moved many products off of the desktop and into the browser. This has greatly reduced the friction involved in releasing a new version of a product. Pushing a code change to servers you manage is a lot easier than persuading your users to download and install an update.
Finally, there has been an architectural shift away from large monolithic systems toward service-oriented architectures and, in recent years, microservices. These modern architectures consist of many small, independently deployable units. This again reduces the friction around deploying a code change by reducing the scope of the deployment to one specific service.
These forces combined have led to software release cycles shortening dramatically, from multiple months or years between each release to a rate of weekly, then daily, then hourly software delivery for some teams. Delivery of a new software release has moved from a marketing event with fanfare to something that is happening almost continuously, even for large products. As far back as 2011, engineers at Amazon were releasing changes to production every 11.6 seconds.
As teams move from a monolith to microservices there is no longer one deployable unit, but rather an ecosystem of independently deployable services. It no longer makes sense to ask which version of a release is running in an environment. Instead, we must ask which version is running for each individual service.
And then there’s feature flagging. One can think of a feature-flagged service as existing in a sort of quantum superposition of potential code paths, resolving to a specific set separately for each request. This means that a service’s behavior in an environment can change radically without any code deployment and without any observable change to the release version.
Now, as we move from continuous delivery to continuous experimentation, we are reaching the point where it is no longer useful to attempt to describe the current running software in terms of a “release,” or even a set of releases. The situation has become a lot more fluid.
Coping in a Post-Release World
Modern approaches like microservices and continuous experimentation are a boon to product delivery teams. They allow us to move faster with more safety, thanks to the tight feedback loops they enable. But this new world also brings challenges.
It can be a lot more difficult to diagnose a bug when you are not sure what code was involved. The load on testers becomes greater as they need to consider the interaction between multiple releases of different services—each with different feature flags which can behave differently from request to request.
From a user’s perspective, a product’s rapid pace of change can be a little overwhelming. New features arriving frequently is nice, but having the UI change every time you log in can be disorienting. Even worse is when a feature is available one day and then gone the next. Users can also be wary of the idea that they are not getting the same experience as other people, particularly if there is any whiff of unfair business practices.
Ironically, while this accelerated feature delivery is partly enabled by continuous experimentation practices, that same speed of change can itself interfere with feature experiments. It can be difficult to statistically correlate an uptick in a KPI with a feature experiment when there are other changes happening over the course of the experiment.
Enhancing the observability of your software—the quality of logging and other instrumentation—is one key way to mitigate some of these challenges. Another is keeping the number of active feature flags reasonably low, which can help on a variety of fronts. It reduces the testing burden of multiple interacting flags, as well as the risk of experiments interfering with one another. Fewer active flags also mean fewer things changing at once for your users.
Managing all feature flags in a shared system can help in understanding the number of active flags and avoiding interfering experiments. Centralized flag management also means you can batch up a set of feature changes into a single release, which helps avoid overwhelming your users with many small changes. A centralized system makes it easier for testers to see which flags they should expect to be in play within different systems, and how those flags might interact.
We’re entering an era that will involve a massive explosion in the rate at which we release new experiences to our users. Embracing this next stage in the evolution of product delivery can enable teams to move faster and with greater safety. But entering this brave new world requires us to develop new practices if we are to avoid the pitfalls of a post-release world.
