AoAD2 Practice: Incremental Requirements

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Note: The “Key Idea: Face-to-Face Communication” sidebar discussed in this excerpt can be found in the “Team Room” practice.

Incremental Requirements

Audience
Customers

We determine requirements details just before they’re needed.

Traditional processes create a requirements document that—in theory—describes exactly how the software is supposed to work. This document is created by business analysts during an up-front requirements gathering phase.

But Agile teams don’t use phases, and story cards aren’t miniature requirements documents. How do they know what to build?

The Living Requirements Document

Written requirements documents introduce delays and errors. First, someone has to take time to create the document; then, some time later, developers have to take time to read and understand the document. It’s hard to write good documents, and even the best documents are easy to misunderstand.

Agile teams prefer to replace work-in-progress documentation with face-to-face communication. (See “Key Idea: Face-to-Face Conversation” on p.XX.) That is, rather than communicating via documents, they communicate via conversation, examples, and whiteboard sketches. This is faster and less error-prone than handing off documents, especially for complex topics. When developers need to understand a requirement, they just ask.

On-site customers—team members with “customer skills” who represent buyers, users, and business stakeholders—are responsible for answering questions about requirements. They act as a living requirements document for the team. They’re expected to figure out requirements just prior to being asked about them.

Allies
Whole Team
Real Customer Involvement
Team Room

The key to successful Agile requirements analysis is expertise. Depending on the needs of your software, your team should include someone with product management skills, who figures out what to build and why; someone with domain expertise, who understands the nitty-gritty of the profession your software supports; someone with user experience design skills, who studies users to understand their work and create productive user interfaces; and maybe even real customers, who provide context about what it’s like to use the software for real. They should each be part of the team room so they can answer questions and provide context as needed.

As an example, for an actuarial product I worked on, our product manager was an actuary, and the sponsors were the senior actuaries for the firm. For a chemical analysis product, the product manager had a Ph.D. in chemistry, we had a dedicated user experience designer, and the domain expert was an analytical chemist.

Don’t rely on your imagination of how your software might be received: go find out.

Even experts need to consider a variety of options and do research before making decisions. Customers, you can and should talk to buyers, users, and other stakeholders. (See “Key Idea: Feedback and Iteration” on p.XX.) Don’t rely on your imagination of how your software might be received: go find out. Ask questions, conduct experiments, and show off working software.

You’re also free to involve other team members. For example, if you’re a user experience designer considering user interface possibilities, discussing them with the team’s programmers can help you strike a balance between an impressive UI and low implementation cost.

Customers decide for themselves how to remember what they’ve learned and decided. (Photos of whiteboard conversations work well.) Whatever you use, treat them as temporary notes. All that matters is that on-site customers can collaborate with other. If you end up needing permanent notes or formal documentation, they come later, as described in “Documentation” on p.XX.

Work Incrementally

Customers, work on requirements incrementally, in parallel with the rest of the team’s work, not in an up-front requirements gathering phase. This makes your work easier and ensures the rest of the team won’t have to wait for requirements analysis to complete before getting started.

Allies
Adaptive Planning

As with the planning horizons of your adaptive plan, you’ll start with the big picture, but not a lot of detail, and then figure out the details of each piece just before they’re needed. On-site customers work together to accomplish this work, typically with product managers focusing on the big picture, and other on-site customers focusing on the details.

Purpose and valuable increments
Allies
Purpose
Visual Planning

Start by clarifying your team’s purpose, then identify valuable increments using visual planning.

The planning game
Allies
The Planning Game

Prior to each planning game session, look at the visual plan and decide which valuable increments will be the subject of the next planning game. Everyone with customer skills should get on the same page about what why those increments are valuable and how you’ll know they’re finished. To save developers’ time, you can optionally break the increments down into smaller stories prior to the planning game, although you should be prepared to make changes during the planning game itself.

During the planning game, developers will ask questions about your expectations for increments and stories. Try to anticipate those questions and have answers ready. (Over time, you’ll learn what sorts of questions developers will ask.) A rough sketch of the visible aspects of the story might help. You may want to collaborate with a few developers in advance to prepare.

Mock-ups, customer examples, and completion criteria
Allies
Customer Examples
Done Done

Figure out the details of each story just before developers start working on it. You should be able to tell when by looking at the visual plan. User experience designers, create mock-ups that show what you expect the work to look like when it’s done. Domain experts, make sure you’re ready to provide examples of tricky domain concepts. Together, decide what it means for each story to be “done.”

Customer review

While stories are under development, before they’re completely done, choose one or more on-site customers to review each story. During the review, check to make sure it works the way you expect. You don’t need to exhaustively test the application—you should be able to rely on developers to test their work—but you should sit with developers and check the areas where developers might think differently than you do. These areas include terminology, screen layout, and interactions between screen elements.

Only working software shows you what you’re really going to get.

Prior to seeing the application in action, every conversation is theoretical. You can discuss options and costs with your developers, but until you have working software, everyone can only imagine how their choices will feel in practice. Only working software shows you what you’re really going to get.

Sometimes, developers will give you exactly what you asked for, but it won’t work as well in practice as you hoped. Other times, there will have been a miscommunication or misunderstanding. In either case, the solution is the same: talk with developers about making changes. You can even pair with developers as they work on fixes.

Allies
Slack

Many changes will be minor, and developers will usually be able to fix them by using the slack in the team’s schedule. If there are major changes, though, they may be too big to fix as part of the current story. This can happen even when the change seems minor from your perspective as a customer. Work with developers to create new stories for those changes. Before scheduling the stories into your visual plan, consider whether the value of the change is worth its cost.

As you continue to collaborate, developers will learn about your expectations for the software. Expect the number of changes you discover to decline over time.

Documentation

Although Agile teams replace work-in-progress documentation with face-to-face communication, some documents still have value. They’re scheduled in the same way as all customer-valued work: with a story. Often, documentation will be just one part of completing a story, but you can also have dedicated documentation stories.

Be careful not to introduce documentation just for the sake of documentation. As with everything the team does, make sure you’re clear about who the documentation benefits and why creating it is more valuable than the other stories you could build.

Product documentation

Product documentation is delivered to customers. Examples include user manuals, help pages, and API reference documentation. One team I worked with packaged up their test results into a formal document that helped their customers pass regulatory approval.

Ally
Done Done

If your software doesn’t have any product documentation, you may still want to document what it does for the team’s future reference. It’s best to do so as part of each story, when people’s memories are fresh, as part of your team’s definition of “done.”

Operations documentation

Operations documentation, also called runbooks, describe standard practices and procedures for a variety of situations, such as deploying the software, responding to outages, provisioning additional resources, and so forth.

Governance documentation

Your organization may require you to create certain documents for governance or auditing purposes. Try to keep this documentation to a minimum, or meet the requirements by creatively repurposing things that have more value. For example, one team used automated acceptance tests, code coverage reports, and source control history to meet a traceability requirement.

Remember that audits don’t necessarily require you to follow a particular process. Often, they only require that you have a process and can prove that you followed it. For example, teams have used pair programming and commit comments to meet a “peer review” auditing requirement. There are often more options for reducing governance documentation than people first imagine.

“As-built” documentation

Agile teams have a rich understanding of what they’re doing and why. But because they don’t create work-in-progress documentation, that understanding vanishes when the team disbands.

As-built documentation is your software’s final increment.

When your team is about to be disband, or move on to a different purpose, take some time to document what you’ve done before moving on. This is the software’s final increment. It’s like packing clothes into mothballs: it doesn’t help you now, but whoever inherits your work will be grateful you took the time. Your goal is to provide an overview that will let them sustain and maintain the code.

As-built documentation can take the form of written documents or video walkthroughs. It generally provides an overview of important concepts, such as architecture, design, and major features. The code and its tests document the details. Alistair Cockburn suggests recording a whiteboard conversation with an eloquent team member explaining the system to a programmer who isn’t familiar with it. Accompany the video with a table of contents that provides timestamps for each portion of the conversation.

Allies
Pair Programming
Collective Code Ownership
Mob Programming

You can postpone the need for as-built documentation by gradually migrating ownership from one team to another. Exploit pair programming and collective code ownership, or mob programming, to migrate new people onto the team and old people off. Instead of a sharp break (or big thud) as one team’s involvement ends and the other begins, the same osmotic communication that helps a team grow can help transition, repopulate, or shrink a team.

Questions

Isn’t it a risk to reduce the amount of documentation?

It could be. In order to reduce documentation, you have to replace it with some other forms of communication. That’s why Agile teams have on-site customers—to replace requirements documents. But you might still need to document what your team has built. If you think that’s valuable, create and prioritize a story for it, or include it in your definition of “done.”

Our customers don’t know what the team should build. What should we do?

Allies
Purpose
Whole Team
Context

Do you have a clear, compelling purpose? If so, your on-site customers should know where to start. If not, your team is missing important customer skills. In this case, you can use traditional requirements gathering techniques, such as [Wiegers 1999], but you’re better off getting people with the skills you need. If you haven’t done so already, try chartering your team’s context and using that to better understand and advocate for the skills you need.

What if a customer review finds too many problems for us to deal with?

This is most likely to happen with a new team, before the team’s developers and customers have learned how to work together. In the short term, you’ll just need to write new stories for the changes needed.

Allies
Mob Programming
Pair Programming

Longer term, customers can spend more time each day talking with developers about their expectations and reviewing in-progress work. Mob programming is the ultimate expression of this idea. Customer/developer pairing is another option. With practice, you’ll learn to anticipate each others’ needs.

As a programmer, I’m offended by some of the things customers find in their reviews. They’re too nitpicky.

Things that can seem nitpicky to programmers—such as the color of the screen background, or a few pixels of alignment in the UI—represent polish and professionalism to customers. This goes both ways: some things that seem important to programmers, such as quality code and refactoring, often seem like unnecessary perfectionism to customers.

Rather than getting upset about these differences of perspective, try to learn what your customers care about and why. As you learn, you will anticipate your customers’ needs better, which will reduce the need to make changes.

Prerequisites

The team must include people who have the time and skill to work out requirements details.
Ally
Whole Team
No Bugs

In order to incrementally define requirements, the team must include people who have the time and skill to work out requirements details. Without them, your team will struggle with insufficient and unclear requirements.

When performing customer reviews, think of them as tools for conveying the customers’ perspective rather than as bug-hunting sessions. Developers should be able to produce code that’s nearly bug-free. The purpose of the review is to bring customers’ expectations and developers’ work into alignment.

Some organizations value written documents so highly that you can’t eliminate work-in-progress documents. Talk with management about why those documents are important and whether direct communication can replace them. Perhaps handoff documents are an acceptable compromise. If not, you don’t have to eliminate the documents, but you do need to include time for them in your planning.

Indicators

When customers work out requirements incrementally:

  • Developers are able to work on established stories while customers figure out the details for future stories.

  • Customers have ready answers to requirements questions, which allows planning and development to proceed quickly and smoothly.

  • By the time a story is done, it reflects customers’ expectations.

  • Customers experience no unpleasant surprises when reviewing software that’s been released.

Alternatives and Experiments

The traditional approach to requirements definition is to perform requirements analysis sessions and document the results. This requires an up-front requirements gathering phase, which takes extra time and introduces communication errors.

Incremental requirements is essentially taking that up-front requirements gathering phase and spreading it out over the entire duration of software development. Rather than customers writing a document, handing it to developers, and then developers reading it, customers speak directly to developers exactly when details are needed.

Ally
Mob Programming

As with stories, people usually experiment with this underlying idea by watering it down. Most often, their team doesn’t include people with customer skills, so they drift back towards a phase-based approach and handing off documents. That decreases agility. If you’re looking for opportunities to experiment, experiment in the other direction: increase communication, increase expertise, and decrease hand-offs. Mob programming is the result of one such experiment.

Further Reading

XXX Thomas Owens recommends Ambler's Agile Modeling

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.