AoAD2 Practice: Incremental Requirements

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Revised: July 14, 2021

Incremental Requirements

Audience
Customers

We determine requirements details just before they’re needed.

Traditional processes create a requirements document which—in theory—describes exactly how the software is supposed to work. This document is created by business analysts during an up-front requirements gathering phase.

But Agile teams don’t use phases, and story cards aren’t miniature requirements documents. How do they know what to build?

The Living Requirements Document

Allies
Whole Team
Real Customer Involvement
Team Room

Agile teams prefer face-to-face communication, as “Key Idea: Face-to-Face Conversation” on page XX discussed. On-site customers—team members with the skill to represent buyers, users, and business stakeholders—are responsible for answering questions about requirements. They act as a living requirements document for the team. They communicate with the rest of the team via conversations, examples, and whiteboard sketches. This is faster and less error-prone than handing off documents, especially for complex topics. When developers need to understand a requirement, they just ask.

On-site customers are expected to figure out requirements just prior to being asked about them. The key to doing this successfully is expertise. Depending on the needs of your software, your team should include someone with product management skills, who figures out what to build and why; someone with domain expertise, who understands the nitty-gritty of the profession your software supports; someone with user experience design skills, who studies users to understand their work and create productive user interfaces; and maybe even real customers, who provide context about what it’s like to use the software to do real work.

As an example, for an actuarial product I worked on, our product manager was an actuary, and the sponsors were the senior actuaries for the firm. For a chemical analysis product, the product manager had a Ph.D. in chemistry, we had a dedicated user experience designer, and the domain expert was an analytical chemist.

Don’t just imagine how your software might be received: go find out.

Even experts need to consider a variety of options and do research before making decisions. Customers, you can and should talk to buyers, users, and other stakeholders. (See “Key Idea: Feedback and Iteration” on page XX.) Don’t just imagine how your software might be received: go find out. Ask questions, conduct experiments, and show off working software.

You’re also free to involve other team members. For example, if you’re a user experience designer considering user interface possibilities, discussing them with the team’s programmers can help you strike a balance between an impressive UI and low implementation cost.

Customers decide for themselves how to remember what they’ve learned and decided. Whatever you use, treat them as temporary notes. All that matters is that on-site customers can collaborate with each other. If you end up needing permanent notes or formal documentation, they come later, as described in “Documentation” on page XX.

When Experts Aren’t Part of the Team

Although Agile teams are supposed to be cross-functional and include people with genuine customer expertise, many organizations have trouble staffing their teams that way. Instead, they choose people to act as intermediaries between developers and people with expertise. For example, a business analyst might be chosen in place of a domain expert, and someone with no decision-making authority might be chosen in place of a product manager.

It’s tempting for people in this situation to become gatekeepers, acting as a go-between and interpreting experts’ statements. Don’t make that mistake. If your on-site customers aren’t experts, then they need to facilitate conversations between the team and the experts, so everyone on the team can learn directly from the source. Then the team works together to incrementally flesh out requirements, as I’ll describe in a moment.

Sometimes, your team will be composed entirely of developers. The situation is similar: some members of the team facilitate conversations with experts, and the whole team works together on requirements.

Work Incrementally

Customers, work on requirements incrementally, in parallel with the rest of the team’s work, not in an up-front requirements gathering phase. This makes your work easier and ensures the rest of the team won’t have to wait for requirements analysis to complete before they get started.

Allies
Adaptive Planning

As with the planning horizons of your adaptive plan, you’ll start with the big picture, but not a lot of detail, and figure out details as they’re needed. On-site customers work together to accomplish this work, typically with product managers focusing on the big picture, and other on-site customers focusing on the details.

Purpose and visual plan
Allies
Purpose
Visual Planning
The Planning Game

Identify the team’s purpose and visual plan from the beginning.

The planning game

Prior to each planning game session, look at the visual plan and decide which increments will be the subject of the next planning game. Everyone with customer skills should get on the same page about what why those increments are valuable and what it means for them to be finished. To save developers’ time, you can optionally break the increments down into smaller stories prior to the planning game, although you should be prepared to make changes during the planning game itself.

During the planning game, developers will ask questions about your expectations for increments and stories. Try to anticipate those questions and have answers ready. (Over time, you’ll learn what sorts of questions developers will ask.) A rough sketch of the visible aspects of the story might help. You may want to collaborate with a few developers in advance to prepare.

Mock-ups, customer examples, and completion criteria
Allies
Customer Examples
Done Done

Figure out the details of each story just before developers start working on it. You should be able to tell when by looking at the visual plan. User experience designers, create mock-ups that show what you expect the work to look like when it’s done. Domain experts, make sure you’re ready to provide examples of tricky domain concepts. Together, decide what it means for each story to be “done.”

Customer review

While stories are under development, before they’re completely done, check to make sure they work the way you expect. You don’t need to exhaustively test the application—you should be able to rely on developers to test their work—but you should sit with developers and check areas where developers might think differently than you do. These areas include terminology, screen layout, and interactions between screen elements.

Only working software shows you what you’re really going to get.

Prior to seeing the application in action, every conversation is theoretical. You can discuss options and costs with your developers, but until you have working software, everyone can only imagine how their choices will feel in practice. Only working software shows you what you’re really going to get.

Sometimes, developers will give you exactly what you asked for, but it won’t work as well in practice as you hoped. Other times, there will have been a miscommunication or misunderstanding. In either case, the solution is the same: talk with developers about making changes. You can even pair with developers as they work on fixes.

Allies
Slack

Many changes will be minor, and developers will usually be able to fix them by using the slack in the team’s schedule. If there are major changes, though, they may be too big to fix as part of the current story. This can happen even when the change seems minor from your perspective as a customer. Work with developers to create new stories for those changes. Before scheduling the stories into your visual plan, consider whether the value of the change is worth its cost.

As you continue to collaborate, developers will learn about your expectations for the software. Expect the number of changes you discover to decline over time.

Documentation

Although Agile teams replace documentation with face-to-face communication, some documents still have value. They’re scheduled with a story, just like any other work. In some cases, updating documentation will be just one part of a larger story, but you can also have dedicated documentation stories, as discussed in “Documentation Stories” on page XX.

Be careful not to introduce documentation just for the sake of documentation. As with everything the team does, make sure you’re clear about who the documentation benefits and why it’s valuable.

Product documentation

Product documentation is delivered to customers. Examples include user manuals, help pages, and API reference documentation. One team I worked with packaged up their test results into a formal document that helped their customers pass regulatory approval.

Ally
Done Done

If your software doesn’t have any product documentation, you may still want to document what it does for the team’s future reference. It’s best to do so as part of each story, when people’s memories are fresh, as part of your team’s definition of “done.”

Operations documentation
Ally
Build for Operation

Operations documentation, also called runbooks, describe standard practices and procedures for a variety of situations, such as deploying the software, responding to alerts and outages, provisioning additional resources, and so forth.

Governance documentation

Your organization may require you to create certain documents for governance or auditing purposes. Try to keep this documentation to a minimum, or meet the requirements by creatively repurposing things that have more value. For example, one team used automated acceptance tests, code coverage reports, and source control history to meet a traceability requirement.

Don’t assume audits require a particular process. They often only require that you have a process—of your choice—and can demonstrate that you followed it. This can give you more options for reducing governance documentation than you think. For example, rather than conducting formal code reviews, teams have used pair programming and commit comments to meet a “peer review” auditing requirement. Talk to audit groups early to build goodwill and create the opportunity for creative solutions.

“As-built” documentation

Agile teams have a rich understanding of what they’re doing and why. But because they don’t communicate via documents, that understanding vanishes when the team disbands.

As-built documentation is your software’s final increment.

When your team is about to be disband, or move to a different purpose, take some time to document what you’ve done. This is your software’s final increment. It’s like packing clothes into mothballs: it doesn’t help you now, but whoever inherits your work will be grateful you took the time. Your goal is to provide an overview that will let them sustain and maintain the code.

As-built documentation can take the form of written documents or video walkthroughs. It generally provides an overview of important concepts, such as architecture, design, and major features. The code and its tests document the details. Alistair Cockburn suggests recording a whiteboard conversation with an eloquent team member explaining the system to a programmer who isn’t familiar with it. Accompany the video with a table of contents that provides timestamps for each portion of the conversation.

Questions

Isn’t reducing the amount of documentation risky?

It could be. In order to reduce documentation, you have to replace it with some other forms of communication. That’s why Agile teams have on-site customers—to replace requirements documents. But you might still need to document what your team has built. If you think that’s valuable, create and prioritize a story for it, or include it in your definition of “done.”

Our customers don’t know what the team should build. What should we do?

Allies
Purpose
Whole Team
Context

Start with a clear, compelling purpose. If your customers don’t know how to pursue that purpose, your team is missing important customer skills. In this case, you can use traditional requirements gathering techniques, such as [Wiegers 1999], but you’re better off getting people with the skills you need. If you haven’t done so already, try chartering your team’s context and using that to better understand and advocate for the skills you need.

What if a customer review finds too many problems for us to deal with?

This is most likely to happen with a new team, before the team’s developers and customers have learned how to work together. In the short term, you’ll just need to write new stories for the changes needed.

Allies
Mob Programming
Pair Programming

Longer term, customers can spend more time each day talking with developers about their expectations and reviewing in-progress work. Mob programming is the ultimate expression of this idea. Customer/developer pairing is another option. With practice, you’ll learn to anticipate each others’ needs.

As a programmer, I’m offended by some of the things customers find in their reviews. They’re too nitpicky.

Things that can seem nitpicky to programmers—such as the color of the screen background, or a few pixels of alignment in the UI—represent polish and professionalism to customers. This goes both ways: some things that seem important to programmers, such as quality code and refactoring, often seem like unnecessary perfectionism to customers.

Rather than getting upset about these differences of perspective, try to learn what your customers care about and why. As you learn, you will anticipate your customers’ needs better, which will reduce the need to make changes.

Prerequisites

Ally
Whole Team
No Bugs
Visual Planning

Incremental requirements requires your team to include people who have the time and skill to work out requirements details. Without them, your team will struggle with insufficient and unclear requirements.

Don’t think of customer reviews as bug-hunting sessions. Developers should be able to produce code that’s nearly bug-free. Instead, the purpose of the review is to bring customers’ expectations and developers’ work into alignment.

Some organizations value written documents so highly, you can’t eliminate requirements documents. Talk with management about why those documents are important and whether direct communication can replace them. Perhaps as-built documentation is an acceptable compromise. If not, include stories for the required documents in your visual plan.

Indicators

When customers work out requirements incrementally:

  • Developers work on established stories while customers figure out the details for future stories.

  • Customers have ready answers to requirements questions, which allows planning and development to proceed quickly and smoothly.

  • By the time a story is done, it reflects customers’ expectations.

Alternatives and Experiments

Incremental requirements essentially takes the traditional up-front requirements gathering phase and spreading it out over the entire duration of software development. Rather than customers writing a document, handing it to developers, and then developers reading it, customers speak directly to developers exactly when details are needed.

Ally
Mob Programming

People usually experiment with this underlying idea by watering it down. Most often, their team doesn’t include people with customer skills, so they drift back towards a phase-based approach and document hand-offs. That decreases agility. If you’re looking for opportunities to experiment, experiment in the other direction: increase communication, increase expertise, and decrease hand-offs. Mob programming is the result of one such experiment.

Further Reading

XXX Thomas Owens recommends Ambler's Agile Modeling

XXX Michal Svoboda suggests: Olsen, D.: The lean product playbook, https://dan-olsen.com/book/

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.