Navigating the Complexity of Hard Choices in Software Development

There are many reasons why software development feels more challenging today than a few decades ago. However, a significant factor is the overwhelming amount of choice across all layers of the tech stack.

Consider all the options available in programming languages, frameworks, databases, tools, platforms, cloud services and more. Not to mention the various architectural styles, system design patterns, agile methodology approaches, etc.

The latest CNCF report and the MAD (ML, AI & Data) Landscape further underscore the dizzying array of choices available to developers.

While we often view having a lot of choice as a good thing, in practice, the allure of unlimited choice can be deceptive.

Although having options gives developers more ways to solve specific project requirements, an excessive number of choices can be overwhelming, leading to indecision, procrastination, suboptimal selections or even complete inaction.

The impact of choice paralysis extends beyond the mental strain on the individual

—which is significant in itself. A compromised decision-making ability impacts a core engineering skill — architecting software — and hinders developers from making their most significant contributions to a project.

Indeed, effective system design hinges on a series of interconnected decisions that define the high-level conceptual structure of a system, including its major components and their interactions. This encompasses all aspects of the system, such as software, hardware, data, interfaces and user interactions.

This blog post aims to explore the factors influencing our choices and to present some strategies for making more informed decisions.

Anatomy of a “Hard Choiceˮ

When A is better than B, the decision is straightforward — you choose A.

When A and B are equivalent, the choice remains simple. If they are equally good, you might as well flip a coin.

However, when A is better than B in some ways, but B is better than A in others, youʼre faced with a hard choice.

The complexity of a “hard choice” arises from how the alternatives compare: in an easy decision, one option is superior, or they are so similar that the choice becomes inconsequential.

In a hard choice, you must perform a trade-off analysis, weighing the pros and cons of each option, knowing that neither is absolutely superior to the other. As philosopher Ruth Chang would describe it, the alternatives are “on a par“: the choice can significantly impact the outcome, yet one option isnʼt universally better than the other.

This scenario is common in software architecture, where there is no single “correct” way to design a system — all choices are valid as long as they meet the product and technical requirements and the team is aware of the trade-offs and potential technical debt.

Randy Shoup, VP of Engineering at eBay and WeWork, captures this sentimen t perfectly: “There is no perfect architecture for all scale, for all phases of evolution, for all problem domainsˮ.

The Layers of Decision-Making

Understanding human decision-making requires considering a multitude of factors, including the physiology of the human brain, psychological influences (such as cultural biases, emotional triggers, past experiences, internal and external expectations and peer pressure) and external environmental factors.

Just to illustrate the complexity of all the factors that influence our choices, consider these studies:

An average human can hold 7±2 items in their short-term memory.
According to Millerʼs Law, on average, weʼre good at comparing one variable at a time with a maximum of 7 choices. For instance, choosing between seven apples based on price alone is manageable. However, our performance decreases as the number of variables — such as color, weight, shape, taste, etc., increases, complicating our ability to perform complex trade-offs and make optimal decisions.
Comparisons change how we evaluate things. Our choices are influenced by the arrangement of options. When selecting wine, a consumer is likely to choose the middle-priced option from a set of three respectively priced at 8$, 27$ and 33$. Introducing a higher-priced fourth option (e.g., 57$ can make the previously highest price of 33$ seem more reasonable, thus shifting consumer preference. Behavioral economist Dan Ariely TED lists other external forces to which weʼre susceptible.
We place a disproportionately high value on things we create ourselves. This phenomenon, known as the “IKEA effectˮ, and explored in a 2011 study, shows that people are willing to pay up to 63% more for furniture they assemble themselves compared to pre-assembled items. However, marketers knew well before this study that “the more effort someone puts into something, the more they will value itˮ. For example, this is the same psychological driver that forced General Mills to revisit their failing “just-add-waterˮ cake mix to “just-add-an-eggˮ. The simple act of replacing the powdered egg with a real egg made the creation of the cake more fulfilling and sales rebounded.
The time of the day affects your choices. This is called the “Hungry judg e effectˮ and it was named after a 2011 study that revealed that judges are more likely to grant parole in the morning than in the afternoon, due to decision fatigue. Although this particular study was later revisited, subsequent research supports the idea that our decision-making capacity diminishes throughout the day, leading to more impulsive and less confident decisions.
When our prefrontal cortex is busy, our emotional brain takes charge. The 1999 “Heart and Mind in Conflictˮ study found that when our prefrontal cortex is overloaded — such as when trying to memorize seven digits — we are more prone to make suboptimal decisions, often favoring emotion-driven decisions (choosing cake over fruit) due to reduced impulse control.
Incorrect incentives might have counterproductive outcomes. The Cobr a effect exemplifies how misguided incentives can backfire. When the British government in colonial India offered a bounty for every dead cobra, it inadvertently encouraged the breeding of snakes for profit, ultimately exacerbating the problem when the program was cancelled and the captive snakes were released. In practice, we might choose an option because of the immediate apparent benefit, even though the end result might be counterproductive for our end goal.

The Paradox of Choice

Barry Schwartz vividly illustrates in his TED Talk, “The Paradox of Choice”, why an abundance of options can lead to decision paralysis and diminish our productivity.

In the era of post-industrial capitalism, the concepts of choice, individual freedom and self-determination have been elevated to an ideal. We often associate freedom with the ability to choose, underpinning the belief that “more choice equals more freedom, which equals more happiness.ˮ

However, this ideology also has a darker side, marked by an increase in anxiety, indecision and perpetual dissatisfaction.

The two primary negative effects of excessive choice are:

1. Choice Paralysis: With a plethora of options available, the decision-making process can become so overwhelming that it becomes difficult to make any choice at all.

2. Dissatisfaction with Decisions: Even after making a choice, we often look back and compare our decision against the options we didn’t select. Each disappointing aspect of our chosen option can breed regret. Moreover, the sheer volume of available choices escalates our expectations for a “perfect” fit, leading inevitably to disappointment when these lofty expectations are not met—even if the outcome is objectively good.

Best Practices for Complex Choices

There are countless strategies, philosophies and books dedicated to enhancing decision-making skills. After all, this skill is crucial not only in software development but in many other disciplines such as finance, research and data analysis.

Here are some best practices that I have personally found effective:

1. Check Your Assumptions Before You Decide

Before making a decision, review and challenge the constraints and requirements that will guide your choice. Ensure there is clear alignment on what is essential versus what is desirable among all stakeholders, and understand the boundaries (financial, regulatory, SLAs, etc.) within which you need to operate. This helps prevent misalignment and the need for backtracking later.

A notable example of the importance of details is the loss of NASA’s $125-millio n Mars Climate Orbiter in 1999 due to a metric conversion error, illustrating how critical accurate information is in decision-making.

2. Employ the Explore-Exploit Trade-off

In their book “Algorithms to Live By: The Computer Science of Human Decisionsˮ Brian Christian and Tom Griffiths propose to use computer strategies to help with decision making.

The explore-exploit trade-off is particularly useful when balancing the choice between trying something new (e.g. a new architectural style) and sticking with a known quantity (e.g. modular monolith).

If facing a tight deadline or a quick release cycle, itʼs advisable to “exploit” by leveraging existing knowledge and choosing the familiar option. Conversely, if the timeframe allows, “exploring” (i.e., experimenting with a new architectural style) may yield valuable insights that improve future decisions.

3. Ask for Support From Your Community

Decision-making does not have to be a solitary task. As psycho-economist Sheena Iyengar points out in her talk “The art of choosingˮ involving your team or community in your decisions can be beneficial. Relying on the judgment of trusted and respected colleagues does not diminish your competence; it underscores the importance of collaboration and recognizes individual limitations—after all, software architecture is a team sport.

4. Donʼt Waste More Time Choosing Than Doing

The 2003 overhaul of the Linux scheduler from an O(n) to an O(1) scheduler illustrates this point beautifully. The former ranked each task individually, often spending more time on task prioritization than on task execution. The switch to the O(1) scheduler that grouped tasks into a limited number of priority buckets increased performance by being less precise but executing in a constant time.

This example highlights that no choice is perfect. Surprisingly, it can sometimes be more effective to approximate a decision and complete a task, moving on to the next one, rather than spending excessive time researching and prioritizing options. Statistically speaking, the optimal stopping point for your research is a t 37% of the way through your options, anyway.

Conclusion

In conclusion, navigating the vast array of choices in software development is no trivial task.

No choice is perfect, and importantly, very few choices are irreversible. Opting for a suboptimal solution or accumulating architectural technical debt simply signals the need for continuous evolution and improvement in your systemʼs architecture. This ongoing process necessitates adapting and making more informed decisions as your system and its requirements evolve.

Itʼs unrealistic to expect that one can consider every option or predict every potential challenge or evolution in requirements. Even adhering to the best decision-making strategy does not guarantee a flawless outcome. A practical approach is to balance the costs of potential errors against the delays that indecision might cause. This mimics the way computers handle complex problems — by simplifying them, removing constraints, or permitting approximations, all aimed at making the most sensible choice in the least amount of time.

Ultimately, the goal is to find a balance that empowers developers to make meaningful decisions without becoming overwhelmed. By implementing strategic decision-making processes, we can provide the necessary framework to reduce choice overload, thus enhancing productivity and satisfaction in software development environments.