Iterative Writing

Behind the scenes of The Art of Agile Development's Pair Programming practice.

I don't think I've talked much about the iterative approach Shane and I used for The Art of Agile Development. Every section received multiple passes, with some being completely reorganized or even rewritten in the process. We posted sections online as soon as they were ready, and some sections got a strong reaction; those were often substantially edited or rewritten, again taking multiple passes. The most controversial got three or four rewrites. The book as a whole was reorganized (refactored!) several times as well, with the outline shifting right up until the book went into production.

Net result? The book took forever! But I'm proud of it.

The "Pair Programming" practice is a particularly good example of how things changed because it's the very first section Shane and I wrote. We used it as a testbed for the rest of the book. We knew that each practice in Part II was going to follow the same basic outline, so we wrote and rewrote (and rewrote, and rewrote, and rewrote) this one until it worked well.

Here's how it went. (Warning: serious navel gazing follows. Unless you're really interested in what it's like to write a book, you should probably stop reading now).

Jan 31, 2006: 1,293 words

This first draft was written in just one day. At this point, we just had "how to" sections and FAQs. Unlike the final book, we didn't start with any justification for why to pair; we just said how. This was partly because we were still planning to create a pocket guide, so we were being very terse.

  • Pair Programming
    • Forming Pairs
    • How to Pair
    • Adopting Pairing
    • FAQs
      • Do we have to pair program all the time?
      • What if we have an odd number of people?

Apr 19, 2006: 2,123 words

Little happened for several months. We added a few notes on why pair programming was good, and a few FAQs, but we were still waiting for our proposal to be accepted. On April 7th, we started an update in a more relaxed style--this must have been around the time we decided to switch from creating a pocket guide to creating a full-fledged book. But despite the large number of new words, the basic structure was unchanged.

I had forgotten this, but apparently we abandoned the "FAQ" header at this point, and just had questions as regular section headers. I'm glad we didn't keep that convention--ick.

  • Pair Programming
    • How to Pair
    • Forming Pairs
    • Adopting Pairing
    • Do we have to pair program all the time?
    • What if we have an odd number of people?
    • There's only two (or three) of us. Should we still pair all the time?
    • Alternatives to Pairing

May 18, 2006: 2,416 words

We still weren't working on the book full time, but we were continuing to make minor changes. The big innovation this time: a dedicated Contraindications section. I'm pretty sure this is when we started standardizing on having Contraindications and Alternatives along with each practice. The Questions section made a comeback, too, thank goodness. This structure is starting to look like the "modern" template we used in the published book.

  • Pair Programming
    • How to Pair
    • Forming Pairs
    • Adopting Pairing
    • Questions
      • Do we have to pair program all the time?
      • What if we have an odd number of people?
      • There's only two (or three) of us. Should we still pair all the time?
    • Contraindications
    • Alternatives to Pairing

Jun 13, 2006: 2,623 words

We started working on the book full-time in June. Pair programming was pretty close to being done (we thought) so we put it up for peer review sometime around the 13th, and incorporated a few comments on the 15th. (I think that's where the new FAQ came from.) For the most part, though, little had changed.

  • Pair Programming
    • How to Pair
    • Forming Pairs
    • Adopting Pairing
    • Questions
      • Do we have to pair program all the time?
      • What if we have an odd number of people?
      • There's only two (or three) of us. Should we still pair all the time?
      • Can we pair remotely?
    • Contraindications
    • Alternatives to Pairing
    • Bibliography

Sep 10, 2006: 2,752 words

That was it for a while. We focused on the rest of the book and made steady progress. Every so often, we'd go back and make more revisions, but the structure remained unchanged. On September 10th, we "finalized" the pair programming section in preparation for sending the whole "Thinking" chapter to our editor. When we did, we added a bunch of FAQs, probably as a result of reviewer comments.

"Finalizing" the chapter was a good thing for the book. We were applying the concept of "Done Done" to the book--trying to create a releasable product, even though we knew there was only going to be one release of the book. By thinking about what we needed to do in order to really release the "Thinking" chapter, we did a lot of the grunt work (like incorporating review comments) and final polish that we could have otherwise put off and possibly forgotten.

Note how the section now has all of the standard section titles in the final book, like "Results," "Alternatives", and "Further Reading." Those standard titles came as a result of this finalization effort.

  • Pair Programming
    • How to Pair
    • Forming Pairs
    • Adopting Pairing
    • Questions
      • How can I convince management to let us pair?
      • Do we have to pair program all the time?
      • What if we have an odd number of people?
      • There's only two (or three) of us. Should we still pair all the time?
      • We get engrossed in our work and forget to switch pairs. How can we encourage more frequent pair switching?
      • If we need to do research, do we continue to pair?
      • My infidel coworkers use emacs, not the One True Editor, vim. What can I do?
      • Can we pair remotely?
    • Results
    • Contraindications
    • Alternatives
    • Further Reading

Nov 6, 2006: 2,798 words

My wife had a baby on September 18th, so we both put the book on hold for six weeks so I could have paternity leave. On November 6th, we generated a PDF of the "Thinking" chapter to send to our editor. Although the structure hadn't really changed (there was one more FAQ), we made some significant changes to some of the wording.

(So much for "finalization." Actually, nothing in the book was really final until we went into production. But the experience of trying to make a chapter "done done" was very healthy.)

Dec 15, 2006: 3,818 words

Our editor got back to us with some good comments, helping us realize that we were using the FAQs as a crutch, and that our text was often too didactic. To fix the problems, we engaged in a major restructuring, and added words to make the content less terse. (It had previously had some echoes of our original, very terse draft.)

We also moved in the material about pairing stations from the "Sit Together" practice, which had gotten way too long. This version was very close to the final version.

  • Pair Programming
    • Why Pair?
    • How to Pair
    • Driving and Navigating
    • Sidebar: Pairing Tips
    • Pairing Stations
    • Challenges
      • Comfort
      • Mismatched Skills
      • Communication Style
      • Tools and Keybindings
    • Questions
      • Isn't it wasteful to have two people do the work of one?
      • How can I convince my team or organization to try pair programming?
      • Do we have to pair program all the time?
      • How can I concentrate with someone talking to me?
      • There's only two (or three) of us. Should we still pair all the time?
      • We get engrossed in our work and forget to switch pairs. How can we encourage more frequent pair switching?
      • Can we pair remotely?
    • Results
    • Contraindications
    • Alternatives
    • Further Reading

Jun 11, 2007: 3,877 words (final version)

We checked in our last changes to the Pair Programming practice on June 11th. Since that rewrite in December, there had only been minor changes.

Throughout this entire process, the most valuable aspect was getting the "Thinking" chapter "done done" and released to our editor. Because each of our practice sections follow a common template, that taught us valuable lessons--such as moving questions into the body of the text where possible, and keeping our writing style light and entertaining--that we were able to carry over to the rest of the book.

If we hadn't focused so much effort on getting the Pair Programming section right, early in the process, I think we would have had a much different book. If we had waited, we would have learned the same lessons (probably, eventually), but I'm not sure that we would have done anything about them. There would have been a lot of "finished" sections to fix and update, and writing the book was a pretty exhausting effort. I don't think we would have had the energy to go back and rewrite them.

The major lesson I took away from this experience is that getting to "done done"--all the way to release--is extraordinarily valuable, just as it is when writing software. There's so much to learn from preparing a real release. For my next book, I'm going to try to get even closer, even earlier.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.