AoAD2 Chapter: Design (introduction)

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Revised: July 20, 2021

Design

Software typically becomes more expensive to change over time.

I’m not aware of any good studies on this,1 but I think it’s something every programmer has experienced. When starting a new codebase, we’re incredibly productive, but as time goes on, changes become more and more difficult.

1The most commonly quoted source is Barry Boehm, who has a chart showing exponential cost increases, but that chart is about the cost of fixing defects by phase, not the costs of change over time. And, as it turns out, that chart doesn’t accurately reflect the underlying data. Laurent Bossavit does an excellent job of chasing down the data in [Bossavit 2013] (chapter 10 and appendix B).

That’s a problem for Agile. If change becomes more significantly more expensive over time, the Agile model doesn’t make sense. Instead, the smart thing to do would be to make as many decisions as possible up-front, when they’re the least expensive. In fact, that’s exactly what pre-Agile methods tried to do.

In order for Agile to work, the cost of change must be relatively flat, or even decreasing over time. Kent Beck discussed this in the first XP book:

[A flat cost of change curve] is one of the premises of XP. It is the technical premise of XP... If a flattened change cost curve makes XP possible, a steep change cost curve makes XP impossible. If change is ruinously expensive, you would be crazy to charge ahead without careful forethought. But if change stays cheap, the additional value and reduced risk of early concrete feedback outweighs the cost of early change. [Beck 2000] (pp. 23-24)

Extreme Programming Explained, 1st ed.

But—as we’ve all experienced—the cost of change isn’t flat. It does increase over time. Does that mean that Agile teams are doomed to collapse under the weight of unmaintainable code?

Without evolutionary design, Agile teams collapse under the weight of unmaintainable code.

The brilliance of XP was that it included practices to proactively reduce the cost of change. Collectively, these practices were called “evolutionary design.” XP remains the only mainstream Agile method to include them. That’s a shame, because without evolutionary design, Agile teams do collapse under the weight of unmaintainable code.

I first heard about evolutionary design in 2000. It sounded ridiculous, but I respected the people recommending it, so I tried an experiment. My team was just about to start a new project. We started with a traditional up-front design, then applied evolutionary design from that point forward.

It worked. It worked incredibly well. Evolutionary design resulted in steady improvement, giving us a design that was cleaner, clearer, and easier to change than the up-front design we started with. I’ve been pushing the boundaries of evolutionary design ever since.

My experience is that evolutionary design does decrease the cost of change over time. I’ve seen it over and over again. As with traditional design, I’m not aware of any good studies on this, but I do have some empirical data to share.

From 2012-2018, I produced a live-coding screencast about evolutionary design and other XP practices. I produced over 600 episodes, resulting in 150 hours of meticulously-documented evolutionary design. The phenomenon I’m talking about—steadily reducing cost of change—happens many times throughout the screencast.2

2The screencast is available at https://www.letscodejavascript.com. The networking example can be found in episodes 370-498 of the Live channel.

One example occurs in the screencast’s implementation of live networking, shown in figure “Real-World Evolutionary Design”. I broke the networking feature up into five stories: First, I networked the user’s mouse pointer, which took 12 hours; then line-drawing, 6½ hours; clearing the screen, 2¾ hours; and two tricky polish stories, ¾ hour and ½ hour. Although the final two polish stories were much more complicated than the initial networking stories, they were also much faster, because they could rely on a clean underlying design.

A graph with two axes. It’s similar to the “cost of change curves” figure, in that it has an x-axis labelled “cost of change” and a y-axis labelled “cost of each change.” The graph has a curve showing an asymptotic decline, and five data points, each representing a change. From left to right, they’re labelled “Pointer (12 hours),” “Lines (6.5 hours),” “Clear button (2.75 hours),” “Disappearing pointer (0.75 hours),” and “Touch pointer (0.5 hours).”

Figure 1. Real-world evolutionary design

This matches my real-world experience. In the real world, each new design challenge goes through the curve shown in the figure, leading to a kind of “saw-tooth” to the cost of change. The overall trend, though, is downhill. With evolutionary design, your design gets steadily better over time, making your software easier and easier to change. The improvements mesh together, too: for example, the final networking changes also benefited from previous design improvements I had made to front-end event-handling code.

Traditional design starts out fast, when everything’s a clean slate. Evolutionary design is the opposite: in the beginning, you’re still feeling your way around, evolving the design as your team has new ideas. But then traditional design slows down and evolutionary design speeds up. After about 4-6 weeks, in my experience, the curves have crossed: code built with evolutionary design is faster and easier to work with than traditional code of the same age. And it keeps getting better.

Evolutionary design is essential to long-term success with Agile.

Evolutionary design is essential to long-term success with Agile. It’s revolutionary. And barely anyone knows about it.

This chapter has three practices for evolutionary design:

  • “Incremental Design” on page XX allows your team to design while they deliver.

  • “Simple Design” on page XX creates designs that are easy to modify and maintain.

  • “Reflective Design” on page XX continuously improves existing designs.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.