Best of 2022: Agile/Scrum is a Failure – Here’s Why

As we close out 2022, we at DevOps.com wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

Welcome to The Long View—where we peruse the news and strip it to the essentials. Let’s work out what really matters.

This week, something a bit different: I’m devoting my entire column to Agile and Scrum. They’re increasingly getting a bad reputation, being associated with the worst aspects of toxic workplace culture: Sexism, racism, burnout, micromanagement.

Analysis: Bad workers blame their tools

Pack up your Post-Its? Skip your Stories? As always, the truth is more nuanced: Agile methods aren’t always the best tools for the job. And when they are, Agile/Scrum require strong, skilled leadership.

What’s the story? Let’s start with Ass. Prof. Miriam Posner’s recent critique—“Agile and the Long Crisis of Software”:

“Exercises in surveillance”
There are distinct limits to the kinds of creativity workers feel authorized to exercise under Agile. … Developers often don’t get to weigh in on the bigger questions. [And] those bigger questions have taken on greater importance and urgency.
…
Agile might have played a role in creating a work culture that is increasingly revealed to be toxic for women, people of color, and members of gender minority groups. … The authors of the Agile Manifesto were … white men who [had] not spent much time in workplaces where they composed the minority. … Eliminating bureaucracy, hierarchy, and documentation feels great, until you’re the person who needs rules for protection.
…
The Agile Manifesto paints an alluring picture. … The problem is, it’s almost always implemented in workplaces devoted to the bottom line, not to workers’ well-being, [such] as when a project manager is caught between a promise to a client and the developers’ own priorities. … Moreover, daily standups … have become, for some workers, exercises in surveillance [with] pressure for every worker to justify their worth.

All of which generated strong opinions, such as this from Xophmeister:

“Building complex software is a social problem”
This is a great summary about how I feel about “Agile”: It’s patronising and anxiety inducing. The degree of which is a function of the technical and social abilities of … the PM, Scrum Master, whatever. … Leaders with strong technical chops and high emotional intelligence exist, but are rare.
…
Similarly, this is why I don’t believe that a fully egalitarian group of engineers would work, either: … Inevitably, a de facto leader will emerge. In my experience, often the loudest, rather than the most effective. Building complex software is a social problem more than a technical one.

And this, by Kokuyo:

“Train wreck”
I am quitting my current job as a private cloud engineer for a cloud provider because they thought it was a good idea to make us work “Agile.” Sure, we are only plannable for 40-60% of our workweek but the problem is … planning even 40% is quite hard when daily business interrupts just about any plan you have.
…
[It’s] led to three burnouts in a team of about eight … two people quitting and two more to come. … I stand there watching the train wreck happening and nobody in management doing anything about it.

Not to mention this, via u/be-sc:

“Went from being agile to implementing Agile”
A company I know used to work closely with its customers: … So it wasn’t a big deal to start out with flawed requirements, because they could iterate quickly towards a workable solution. Overall it was a pretty agile way of working—although nobody thought of it like that, it was just the natural thing to do.
…
Then the world changed and things were introduced in the name of becoming Agile and practicing continuous improvement. Think of stuff like SAFe or SPICE. And SCRUM on top of it, of course. As a result, those quick efficient iterations aren’t officially allowed any more. Everything needs to be planned and documented and tracked and reviewed and approved and documented again. Meetings need to be held on all of that, of course. And, oh, this only looks like waterfall, but it totally isn’t.
…
The company went from being agile to implementing Agile—and lost its agility.

Ah yes, Waterfall by any other name. Or, as Jeremy Duvall puts it, “Scrumfall”:

“Chaos, crappy code, cost overruns”
The Church of Agile is being corrupted from within by institutional forces that [can’t] adapt to the radical humanity [of] collaborative, self-organizing, cross-functional teams. … Agile wasn’t supposed to be this way.
…
Agile is supposed to be centered on people, not processes. … But many businesses instead prioritize controlling their commodity human resources. … Companies have dressed it up in Scrum’s clothing, claiming Agile ideology while reasserting Waterfall’s hierarchical micromanagement. … Properly implemented Scrum or Kanban [should] lead to the desired outcome within finite time and budget.
…
Stories as mini-Waterfalls [treat] the engineer as a cog in their employer’s machine … with no understanding of the craft, creativity, and critical thinking required to solve such complex problems. … Scrumfall relies, in other words, on the product team … providing a complete and perfect specification before development begins. And it relies on the development team … planning out a complete and perfect implementation before a single line of code is written.
…
The invading Waterfall taskmasters hidden in Scrum’s Trojan Horse absolutely hate uncertainty. However, uncertainty is not a bug but a feature of Agile. [But] these chains of mini Waterfalls create chaos, crappy code, and, ironically, cost overruns and missed deadlines.

But beware of zealotry. Heed the wise words of cestith:

“Anything larger than a developer can do in two weeks is infeasible”
One of the problems people are having is that so much is said against Waterfall instead of just in favor of Agile. … People emphasize that everything to do with Waterfall is bad, and generalize from large, detailed, inflexible architecture with no feedback loops being bad. [This is] repeated by people trained to be Scrum Masters or some other management or administrative role who are not developers themselves.
…
I’ve been told that gating one story behind another is waterfall. I’ve been told that having a design story is waterfall. I’ve been told that architecture documents that are eligible to be revised … but created before all the code is in place are waterfall.
…
Taken to heart [it] means that anything larger than a developer can do in two weeks is infeasible. This often happens when the team building the software is not considered [as] stakeholders.

Whatever happened to “people over process”? Slicker gets real:

“It just doesn't work that way”
Ethics were never a part of software development methods. Those are business decisions. It’s not Agile’s fault or Waterfall’s fault. A hammer is a hammer: You can drive a nail into wood or clobber somebody.
…
As far as giving engineers autonomy, I do see enormous largely unrecognized value in that. Boeing, for example, made great products until the management style was changed to a top-down authoritarian model. … Obviously quality, productivity, and innovation all suffer horrendously. … And I have seen this pattern in company after company.
…
Engineers need latitude to solve problems. Their work is nothing like a production line in a factory. It just doesn’t work that way.

As in other walks of life, there’s no single answer—and nuance is essential. Here’s u/Sammy81:

“You have to plan the whole project”
It comes down to: What are you developing? Some products are well suited to Agile and others aren’t. “Pure” agile is best for a commercial product. … You have a feature set that you prioritize and you get through as much as you can before you run out of time, call it done and try to sell it.
…
When you have a customer who has a list of requirements they insist on, … you can’t just do … sprints and see how far you get. … You have to plan the whole project, resulting in Agilefall.

Meanwhile, I think we can all agree it’s the product manager’s fault. Amirite, spaetzleesser?

I have never seen one that could really guide a project from start to finish. They either don’t understand the tech sufficiently, or they don’t understand the business case, or they aren’t engaged enough.
…
[Or,] once a product manager has enough experience to understand the business and the tech, they get promoted and you have to deal with a newbie.

The Moral of the Story:
The fault is not in our stars—but in ourselves

You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

Image: Brad West (via Unsplash; leveled and cropped)