This is an excerpt from The Art of Agile Development, Second Edition. Visit the Second Edition home page for additional excerpts and more!
This excerpt is copyright 2007, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.
Software typically becomes more expensive to change over time.
I’m not aware of any good studies on this,1 but I think it’s something every programmer has experienced. When starting a new codebase, we’re incredibly productive, but as time goes on, changes become more and more difficult.
1The most commonly quoted source is Barry Boehm, who has a chart showing exponential cost increases, but that chart is about the cost of fixing defects by phase, not the costs of change over time. And, as it turns out, that chart doesn’t accurately reflect the underlying data. Laurent Bossavit does an excellent job of chasing down the data in [Bossavit2013] (ch. 10 and appendix B).
That’s a problem for Agile. If change becomes significantly more expensive over time, the Agile model doesn’t make sense. Instead, the smart thing to do would be to make as many decisions as possible up front, when they’re the least expensive. In fact, that’s exactly what pre-Agile methods tried to do.
For Agile to work, the cost of change must be relatively flat, or even decreasing over time. Kent Beck discussed this in the first XP book:
[A flat cost of change curve] is one of the premises of XP. It is the technical premise of XP...If a flattened change cost curve makes XP possible, a steep change cost curve makes XP impossible. If change is ruinously expensive, you would be crazy to charge ahead without careful forethought. But if change stays cheap, the additional value and reduced risk of early concrete feedback outweighs the cost of early change. [Beck2000a] (ch. 5)
Extreme Programming Explained, 1st edition
But—as we’ve all experienced—the cost of change isn’t flat. It does increase over time. Does that mean that Agile teams are doomed to collapse under the weight of unmaintainable code?
Without evolutionary design, Agile teams collapse under the weight of unmaintainable code.
The brilliance of XP was that it included practices to proactively reduce the cost of change. Collectively, these practices were called “evolutionary design.” XP remains the only mainstream Agile method to include them. That’s a shame, because without evolutionary design, Agile teams do collapse under the weight of unmaintainable code.
I first heard about evolutionary design in 2000. It sounded ridiculous, but I respected the people recommending it, so I tried an experiment. My team was just about to start a new project. We started with a traditional up-front design, then applied evolutionary design from that point forward.
It worked. It worked incredibly well. Evolutionary design resulted in steady improvement, giving us a design that was cleaner, clearer, and easier to change than the up-front design we started with. I’ve been pushing the boundaries of evolutionary design ever since.
My experience is that evolutionary design does decrease the cost of change over time. I’ve seen it over and over again. As with traditional design, I’m not aware of any good studies on this, but I do have some empirical data to share.
From 2012–2018, I produced a live-coding screencast about evolutionary design and other XP practices. I produced over 600 episodes, resulting in 150 hours of meticulously documented evolutionary design. The phenomenon I’m talking about—steadily reducing cost of change—happens many times throughout the screencast.2
One example occurs in the screencast’s implementation of live networking, shown in the “Real-World Evolutionary Design” figure. I broke the networking feature up into five stories: first, I networked the user’s mouse pointer, which took 12 hours; then line-drawing, 6½ hours; clearing the screen, 2¾ hours; and two tricky polish stories, ¾ hour and ½ hour. Although the final two polish stories were much more complicated than the initial networking stories, they were also much faster, because they could rely on a clean underlying design.
This matches my real-world experience. In the real world, each new design challenge goes through the curve shown in the figure, leading to a kind of “saw-tooth” appearance to the cost of change. The overall trend, though, is downhill. With evolutionary design, your design gets steadily better over time, making your software easier and easier to change. The improvements mesh together, too: for example, the final networking changes also benefited from previous design improvements I had made to frontend event-handling code.
Traditional design starts out fast, when everything’s a clean slate. Evolutionary design is the opposite: in the beginning, you’re still feeling your way around, evolving the design as your team has new ideas. But then traditional design slows down and evolutionary design speeds up. After about 4–6 weeks, in my experience, the curves have crossed: code built with evolutionary design is faster and easier to work with than traditional code of the same age. And it keeps getting better.
Evolutionary design is essential to long-term success with Agile.
Evolutionary design is essential to long-term success with Agile. It’s revolutionary. And barely anyone knows about it.
This chapter has three practices for evolutionary design:
The “Incremental Design” practice allows team members to design while they deliver.
The “Simple Design” practice creates designs that are easy to modify and maintain.
The “Reflective Design” practice continuously improves existing designs.
The practices in this chapter are based on Extreme Programming’s approach to design. In the conversation surrounding XP, it was called “evolutionary design,” which is the umbrella term I’m using here. Kent Beck called it Simple Design in the first edition of XP [Beck2000a] and Incremental Design in the second [Beck2004].
I’ve used both of Beck’s terms for the practices in this chapter, each focusing on a different aspect of evolutionary design, and added Reflective Design, which was called “merciless refactoring” in the early days of XP.
The details I discuss in each practice come from a variety of sources, as well as my own experiences with evolutionary design. Martin Fowler and XP’s “Three Amigos”—Kent Beck, Ron Jeffries, and Ward Cunningham—were all major influences.