AoAD2 Practice: No Bugs

This is an excerpt from The Art of Agile Development, Second Edition. Visit the Second Edition home page for additional excerpts and more!

This excerpt is copyright 2007, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

No Bugs

Whole Team

We release with confidence.

If you’re on a team with a bug count in the hundreds or thousands, the idea of “no bugs” probably sounds ridiculous. I’ll admit it: no bugs is an ideal to strive for, not something your team will completely achieve. There will always be some bugs. (Or defects; I use “bug” and “defect” interchangeably.)

But you can get closer to the “no bugs” ideal than you might think. Consider Nancy van Schooenderwoert’s experience with Extreme Programming. She led a team of novices working on a real-time embedded system for farm combines: a concurrent system written in C, with some assembly. If that’s not a recipe for bugs, I don’t know what is. According to her analysis of data by Capers Jones, the average team developing this software would produce 1,035 defects and deliver 207 to the customer.

Here’s what actually happened:

The GMS team delivered this product after three years of development, having encountered a total of 51 defects during that time. The open bug list never had more than two items at a time. Productivity was measured at almost three times the level for comparable embedded software teams. The first field test units were delivered after approximately six months into development. After that point, the software team supported the other engineering disciplines while continuing to do software enhancements. [VanSchooenderwoert2006]

“Embedded Agile Project by the Numbers with Newbies”

Over three years, the team generated 51 defects and delivered 21 to its customer. That’s a 95% reduction in generated defects and a 90% reduction in delivered defects.

We don’t have to rely on self-reported data. QSM Associates is a well-regarded company that performs independent audits of software development teams. In an early analysis of a company practicing a variant of XP, they reported an average reduction from 2,270 defects to 381 defects, an 83% decrease. Furthermore, the XP teams delivered 24% faster with 39% fewer staff. [Mah2006]

More recent case studies confirmed those findings. QSM found 11% defect reduction and 58% schedule reduction on a Scrum team; 75% defect reduction and 53% schedule reduction on an XP team; and 75% defect reduction and 30% schedule reduction in a multiteam analysis of thousands of developers. [Mah2018]

Eliminate errors at their source rather than finding and fixing them after the fact.

How do you achieve these results? It’s a matter of building quality in, rather than testing defects out. Eliminate errors at their source rather than finding and fixing them after the fact. continue reading, buy the book!

In this Section

  1. No Bugs
    1. Key Idea: Build Quality In
    2. Don’t Play the Bug Blame Game
    3. How to Build Quality In
      1. Prevent programmer errors
      2. Prevent design errors
      3. Prevent requirements errors
      4. Prevent systemic errors
    4. Fix Bugs Immediately
    5. Testers’ Role
    6. ’Tude
    7. Questions
    8. Prerequisites
    9. Indicators
    10. Alternatives and Experiments

Discuss the book on the AoAD2 mailing list or Discord server. For videos and interviews regarding the book, see the book club archive.

For more excerpts from the book, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.