Print

Singed Egos

16 Jul, 2008

Behind the scenes of The Art of Agile Development's No Bugs practice.

"No Bugs" may be my favorite section of The Art of Agile Development. It describes, in clear and pragmatic terms, exactly how all of the practices in the book fit together to achieve high quality and low defects.

It didn't start out so strong. Actually, it was terrible. The practice was originally titled "No Bug Database." We got more flack on this section than any other. At one point, the Joel on Software forums picked up on the material, and flamed us thoroughly. (In three parts: one, two, three.)

It took a little while to get over the singed egos, but once we did, we were able to use the feedback to create a much stronger result. We still got flamed in the forums, but they went from "you're obviously stupid," to "I'm not convinced." By Internet forum standards, that's high praise.

I think I've mentioned before that we had an open review process, with every section of the book posted on the web for public comment. (We ended up with over 1,000 messages on our review mailing list.) This section is a great example of how important that review process was to the quality of the final book. To illustrate, I'd like to share our initial draft so you can compare it with the final manuscript. This was the "final" draft at the time. This would have gone in the book if it hadn't been for the feedback.


No Bug Database

Audience
Whole Team

Fix bugs now.

Here's a crazy idea. Get rid of your bug database. No, I'm not advocating that you ignore problems. Exactly the opposite. Hear me out.

Bug databases are often a black hole of to-dos, packed with hundreds of questions, feature requests, and genuine defects. Beyond about ten items, who has the time or energy to slog through that mess? You don't have to keep that weight around your neck. You already have a better way to handle bugs and feature requests.

A Better Way to Fix Bugs

When you find a bug, fix it.

What if you had a zero-tolerance policy toward bugs? Programmers, what if you fixed bugs as soon as you found them, rather than filing bug reports? You don't have to worry about who originally owned the code (section X.Y, "Shared Code"). You can even do some root-cause analysis (section X.Y, "Root Cause Analysis") and refactor the code (section X.Y, "Refactoring") to make similar bugs impossible.

The same idea goes for customers and testers. When you find or hear about a bug, what if you took ownership of it? What if you figured out how to reproduce it, found a free programmer, and helped him fix it?

A Better Way to Track Requests

Of course, sometimes you're in the middle of an important task and need to finish it before fixing the bug. My preferred tracking mechanism is for the navigator to write the problem on our to-do list and then come back to it as soon as we get to a stopping point.

If you have too little time to fix bugs as they come up, you may need more slack (section X.Y, "Slack") in your schedule.

Some bugs are too big to identify and fix immediately. Others are feature requests in disguise. For these bugs, you need to track them for future scheduling. XP teams already have a way to prioritize and schedule work: stories (section X.Y, "Stories") and release planning (section X.Y, "Release Planning").

When you find something that you can't fix right away, create a story card for it. Add it to the stack of pending stories as usual, then let the product manager and other on-site customers schedule it as with any other story.

How Can This Possibly Work?

Can bug tracking really be this simple? How is it possible to eliminate a bug database? There are so many teams with hundreds or even thousands of items in their defect tracking system. If it were so simple, wouldn't they be doing it, too?

The more quickly you fix a bug, the cheaper it is to fix.

The key is high code quality—leading to fewer bugs in the first place—and rapid response to defects. The longer a defect sits in a system, the more expensive it is to fix [Boehm], and, we believe, to analyze for feedback. Ironically, the existence of a bug database is an invitation to ignore problems—to write them down so they'll become somebody else's problem—rather than fixing them. This raises costs.

Eliminating the bug database makes defect elimination everyone's responsibility. When you find a bug, you can't just take 30 seconds to type in an explanation... you have to go fix it. By doing so, you've shrunk the time between bug insertion and bug fix, which reduces the cost of fixing defects and helps keep code quality high by letting you learn from mistakes as soon as possible. It's a virtuous cycle that decreases the number of defects, making rapid response easier, which increases code quality, which in turn decreases the number of defects.

Success with this approach requires discipline and a passionate desire to eliminate bugs as soon as they occur. Your whole team must share this goal. If people take an "out of sight, out of mind" attitude instead, it won't work.

Questions

Do we really fix all bugs right away?

Immediately fix the bugs that need fixing. The on-site customers may decide that some bugs aren't important enough to fix now, if ever—usually the ones that are low impact and expensive to fix. This is a sensible quality/time tradeoff, albeit one to make with care.

This is silly. Has anybody really done this?

It seems pretty radical, doesn't it? Think about it, though: what's the difference between fixing bugs when you find them and batching them up to fix at the end of the project? You have to fix those bugs eventually, and it's cheaper to fix bugs sooner, so why wait? You'll reduce the chance of other people running into the same bugs (and trawling through your reporting system, filing duplicate reports) and have the chance to identify and eliminate the root causes for these bugs too.

[Schooenderwoert] tells a story of an embedded, multi-threaded C project composed primarily of novice developers (emphasis added):

The GMS team delivered this product after three years of development, having encountered a total of 51 defects during that time. The open bug list never had more than two items at a time. Productivity was measured at almost 3 times the level for comparable embedded software teams. The first field test units were delivered approximately six months into development. After that point, the software team supported the other engineering disciplines while continuing to do software enhancements.

If they can do it, so can you.

How long should we spend working on a bug before make it into a story?

It depends on how much slack you have left in the iteration. Early in the iteration, when there's still a lot of slack, I might spend as much as four pair-hours on a defect. Later, when there's less, I might only spend ten minutes on it.

Bugs are usually harder to find than to fix, so enlist the help of the testers. The fix often takes mere minutes once you've isolated it.

If we're supposed to fix bugs as soon as we find them, won't we be tempted to ignore them?

Perhaps. This is why we pair: pairing helps us maintain our team discipline. If you find yourself succumbing to the temptation to ignore a bug, write it on a story or task card rather than letting it slide by. Let the rest of the team know about the bug and ask somebody to volunteer to fix it.

We already have a bug database. How should we handle it?

You can stop using the bug database for new issues and just use it for old ones. Alternatively, customers and testers can spend some time going through the database, eliminating duplicates and unimportant issues, and turning the rest into story cards.

Stakeholders use our bug database. What should we tell them?

You may wish to leave that database in place. A stakeholder-facing bug database or issue tracker can be a convenient way to gather stakeholder feedback. Every day or so, have the testers and on-site customers sort through the outstanding issues. Give any real bugs the zero-tolerance treatment by fixing them right away. Discuss the remaining issues with stakeholders and turn them into stories as appropriate. Whittle the outstanding issues down to zero every time. Don't let them pile up.

Our on-site customers like to use a bug database for holding more information about stories. Is that okay?

Sure. Story cards are just for prioritization; customers often need to hold additional information somewhere. Some teams use a bug database or issue tracker. Others use a Wiki and still others use spreadsheets and PowerPoint. The customers should use whatever they're comfortable with—just be sure to use it as an aid to planning, not a replacement for it, and don't use it to track actual bugs.

If we don't have a bug database, won't people report the same issues over and over?

In my experience, they do that even if you do have a bug database. Of course, if you fix the bug right away and release to production frequently (section X.Y, "Release Planning"), there won't be anything to report.

Contraindications

All you need in order to follow this practice is a burning desire to eliminate bugs as soon as they appear. Although it helps if your team doesn't generate very many bugs, that isn't necessary.

If your team generates a lot of bugs, you will probably need to fix them eventually. Why not now? Yes, your velocity will go down because you're fixing bugs rather than adding features. That's okay—you'll probably release a little sooner overall and it's better to have a known, predictable hit to your velocity now than it is to have an indeterminate number of bug-fix weeks before you can release.

To succeed without a bug database, the team has to be disciplined about fixing bugs rather than ignoring them. Pair programming (section X.Y, "Pair Programming") helps reduce the temptation to ignore problems. The XP development practices (section X.Y, "Developing") also help by decreasing the number of bugs your team creates in the first place. A whole team (section X.Y, "Whole Team") that sits together (section X.Y, "Sit Together") is also helpful, as it allows you to squash tough bugs more quickly by attacking them as a team.

Alternatives

You can use a bug database if you want. Ultimately, the question of whether to fix a bug now, later, or never is the product manager's responsibility. Every issue that affects the schedule and scope of the project in a significant way (that is, by more than a few hours) needs to make it to a story card somehow.

If you use a dedicated bug database, you need some mechanism for getting stories out of the bug database and into the release plan (section X.Y, "Release Planning"). Try not to let the bugs pile up, as that will give you a false impression of your velocity and what you can release.

Further Reading

"Embedded Agile Project By the Numbers with Newbies" [Schooenderwoert] reports on the author's experience in more detail.


Thank goodness for feedback.