As more organizations use feature flags and experiments, it’s critical to understand that some of these changes should exist for only a short or specific period of time, and then be removed from your codebase. Removing flags when appropriate avoids technical debt and complexity that can make your code fragile, harder to test and more difficult to support.
The simplest ways to evaluate when a feature flag or experiment should be removed include looking at how long it’s been in the system and how many different states it’s in. If the feature flag has been in your system for a long time and all of your users have the same state of the feature flag, then it should likely be removed.
However, evaluating based on just those two factors isn’t enough. It’s smart to also evaluate the purpose of a flag or experiment before finalizing the decision to remove it. The appropriate lifetime of a feature flag or experiment, and when it should be removed from your codebase, depends heavily on its use case.
When to Remove Temporary Flags and Experiments
If a feature is designed to be rolled out to everyone and you don’t expect to experiment on the feature once it’s launched, then you’ll want to ensure you have a ticket tracking its removal as soon as the feature has been fully launched to your users. These temporary flags may last weeks, months or quarters. Examples include:
- Painted-door experiments: These experiments are intended to be used only in the early phases of the software development lifecycle and aren’t intended to be kept in the product after they have validated or invalidated the experiment hypothesis.
- Performance experiments: These experiments are intended to put two different implementations against each other in a live, real-world performance experiment. Once enough data has been gathered to determine the more performant solution, it’s usually best to move all traffic over to the higher-performing variation and remove the related flag.
- Large-scale refactors: When moving between frameworks, languages or implementation details, it’s useful to deploy these rather risky changes behind a feature flag so that you have confidence they won’t negatively impact your users or business. However, once the re-factor is done, you hopefully won’t go back in the other direction.
- Product re-brands: If your business decides to change the look and feel of your product for brand purposes, it’s useful to have a gradual rollout to gracefully move to the new branding. After the new branding is established, it’s a good idea to remove the feature flag powering the switch.
How to Review the Lifetime of Permanent Flags and Experiments
If a feature is designed to have different states for different customers, or if you want to control its configuration for operational processes, then it’s likely the flag will stay in your product for a longer period of time. Examples of these flags and experiments include:
- Permission flags: These flags are useful if you have different permission levels in your product, such as “read-only” that doesn’t allow edit access to the feature. They are also useful if you have modular pricing like an inexpensive “starter plan” that doesn’t have the feature, but a more costly “enterprise plan” that does have the feature.
- Operational flags: These flags control the operational knobs of your application. For example, these flags can control whether you batch events sent from your application to minimize the number of outbound requests. You could also use them to control the number of machines that are used for horizontally scaling your application. In addition, they can be used to disable a computational expensive, non-essential service or allow for a graceful switchover from one third-party service to another in an outage.
- Configuration-based software: For any software or product that is powered by a configuration file, this is a great place to seamlessly insert experimentation that has a low cost to maintain and still allows endless product experimentation. For example, some companies may have their product layout powered by a config file that describes in abstract terms whether different modules are included and how they are positioned on the screen. With this architectural setup, even if you aren’t running an experiment right now, you can still enable future product experimentation.
Note that even if a flag is meant to be permanent, it’s still critical to regularly review these flags in case they become obsolete or should be deprecated, which would then add technical debt to your codebase.
Establishing a Feature Management Process
Creating an always-on feature management process goes hand-in-hand with using feature flags and experimentation effectively. Some organizations use a combination of a task tracking system and their feature flag and experiment platform to manage this cleanup process seamlessly and quickly. If the state of feature flags and experiments can be synced with a ticket-tracking tool, then an engineering manager can run queries for all feature flags and experiments whose state has not been changed in the past 30 days, and track down owners of the feature flags and experiments to evaluate their review. Other teams schedule recurring feature flag and experiment removal days to ensure engineers review the oldest items on a regular basis to minimize technical debt.
The benefits of feature flags and experimentation to help teams ship code and launch new features quickly and safely far outweigh the needed management to minimize codebase technical debt and complexity. When managed as a recurring process, looking at all key factors (lifetime, state and use case), teams can feel confident that their systems will remain efficient and reliable.