AoAD2 Practice: Incremental Requirements

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Note: The “Key Idea: Face-to-Face Communication” sidebar discussed in this excerpt can be found in the “Team Room” practice.

Incremental Requirements

Audience
Customers

We define requirements in parallel with other work.

Traditional processes provide a “requirements document” to developers that describes exactly how the software is supposed to work. This document is created by business analysts during an up-front “requirements gathering” phase.

But Agile teams don’t use phases, and story cards aren’t miniature requirements documents. How do they know what to build?

The Living Requirements Document

Allies
Whole Team
Team Room

On-site customers—team members with the skills to represent customers, users, and other business stakeholders—act as “living requirements documents” for the team. They’re expected to have all the information about requirements at their fingertips. When somebody on the team needs to know something about requirements, they ask one of the on-site customers rather than looking in a document.

Face-to-face communication is much more effective than written communication.

Face-to-face communication is much more effective than written communication, as “Key Idea: Face-to-Face Conversation” discusses. By using communicating directly, Agile teams avoid the delay of an up-front requirements phase and the misunderstandings caused by written documentation. However, requirements work is still necessary. On-site customers still need to understand requirements before they can explain them.

Allies
Real Customer Involvement

The key to successful Agile requirements analysis is expertise. Your team should include an experienced product manager, experts in your problem domain, and maybe even real customers. If usability is important, include a professional user experience designer.

As an example, for an actuarial product I worked on, our product manager was an actuary, and the sponsors were the senior actuaries for the firm. For a chemical analysis product, the product manager had a Ph.D. in chemistry, we had a dedicated user experience designer, and the domain expert was an analytical chemist whose job had centered around using software like ours.

Some requirements will necessitate even expert customers to consider a variety of options, or do some research, before making a decision. Customers, you can and should talk to real customers, users, and other stakeholders. You’re also free to involve other team members. For example, if you’re a user experience designer considering user interface possibilities, discussing them with the team’s programmers can help you strike a balance between an impressive UI and low implementation cost.

Keep track of requirements using whatever lightweight notes you prefer.

Keep track of requirements using whatever notetaking techniques you prefer. They’re not a formal requirements document, so you can keep them light and informal. Any notes you produce are for your own personal use, and perhaps the use of the other customers on the team, depending on how you decide to work together. If you end up needing to create something formal, it will come later.

Work Incrementally

Allies
Adaptive Planning

Work on requirements incrementally, in parallel with the rest of the team’s work. This makes your work easier and ensures the rest of the team won’t have to wait for requirements analysis to complete before getting started. Your work will typically parallel the release planning horizons of your adaptive plan. As you work, be sure to coordinate with the other customers on the team.

Vision and SIVs
Allies
Purpose
Visual Planning

Start by clarifying your team’s purpose, then identify shippable increments of value (SIVs) using visual planning. These initial ideas will guide the rest of your requirements work.

The planning game
Allies
The Planning Game

Prior to each planning game, look at the release planning board and consider the SIVs that will part of the next the planning game. Decide what those SIVs mean to you and how you’ll know they’re finished. If you like, you can break the SIVs down into stories prior to coming to the planning game, although you should be prepared to make changes during the planning game itself.

During the planning game, developers will ask questions about your expectations for SIVs and stories. Try to anticipate those questions and have answers ready. (Over time, you’ll learn what sorts of questions developers will ask.) A rough sketch of the visible aspects of the story might help. You may want to collaborate with a few developers in advance to prepare.

Mock-ups, customer examples, and completion criteria
Allies
Customer Examples
Done Done

You don’t need to figure out the details for each story until just before developers start implementing it. Create mock-ups that show what you expect the work to look like when it’s done. Prepare examples of tricky domain concepts and decide what “done” means for each story. You’ll typically do this work just a few days before developers start on the story. You can see which stories are coming up next by looking at the release planning board.

Customer review

While stories are under development, before they’re completely done, review each story to make sure it works the way you expect. You don’t need to exhaustively test the application—you should be able to rely on developers to test their work—but you should sit with developers and check the areas where developers might think differently than you do. These areas include terminology, screen layout, and interactions between screen elements.

Only working software shows you what you’re really going to get.

Prior to seeing the application in action, every conversation is theoretical. You can discuss options and costs with your developers, but until you have working software, everyone can only imagine how their choices will feel in practice. Only working software shows you what you’re really going to get.

Sometimes, developers will give you exactly what you asked for, but it won’t work as well in practice as you hoped. Other times, there will have been a miscommunication or misunderstanding. In either case, the solution is the same: talk with developers about making changes. You can even pair with developers as they work on fixes.

Allies
Slack

Many changes will be minor, and developers will usually be able to fix them by using the slack in the team’s schedule. If there are major changes, though, they may be too big to fix as part of the current story. This can happen even when the change seems minor from your perspective as a customer. Work with developers to create new stories for those changes. Before scheduling the stories into your visual plan, consider whether the value of the change is worth its cost.

As you continue to collaborate, developers will learn about your expectations for the software. Expect the number of changes you discover to decline over time.

Documentation

Allies
Team Room
Customer Examples
Stories

Agile teams prefer to replace work-in-progress documentation with face-to-face communication. That is, rather than communicating via documents, they communicate via conversation, examples, and whiteboard sketches. This is faster and more effective than handing off documents, especially for complex topics.

That said, some documents still have value. The main types of documentation Agile teams produce are product documentation, operations documentation, and hand-off documentation. They’re scheduled in the same way as all customer-valued work: with a story. Often, documentation will be just one part of completing a story, but you can also have dedicated documentation stories.

Product documentation

Product documentation is delivered to customers. Examples include user manuals, help pages, and API reference documentation. One team I worked with packaged up their test results into a formal document that helped their customers pass regulatory approval.

Ally
Done Done

If your software doesn’t have any product documentation, you may still want to document what it does for the team’s future reference. It’s best to do so as part of each story, when people’s memories are fresh, as part of your team’s “definition of done.”

Operations documentation

Operations documentation, also called runbooks, describe standard practices and procedures for a variety of situations, such as deploying the software, responding to outages, provisioning additional resources, and so forth.

Hand-off documentation

Agile teams have a rich understanding of what they’re doing and why. But because they don’t create work-in-progress documentation, that understanding vanishes when the team disbands.

Handoff documentation is your software’s final SIV.

When your team is about to be disband, or move on to a different purpose, take some time to document what you’ve done before moving on. This is the software’s final SIV. It’s like packing clothes into mothballs: it doesn’t help you now, but whoever inherits your work will be grateful you took the time. Your goal is to provide an overview that will let them sustain and maintain the code.

Allies
Pair Programming
Collective Code Ownership
Mob Programming

As an alternative to handoff documentation, you can gradually migrate ownership from one team to another. Exploit pair programming and collective code ownership, or mob programming, to migrate new people onto the team and old people off. Instead of a sharp break (or big thud) as one team’s involvement ends and the other begins, the same osmotic communication that helps a team grow can help transition, repopulate, or shrink a team.

Questions

Isn’t it a risk to reduce the amount of documentation?

It could be. In order to reduce documentation, you have to replace it with some other forms of communication. That’s why Agile teams have on-site customers—to replace requirements documents. But you might still need to document what your team has built. If you think that’s valuable, create a story for it and prioritize it during the planning game.

Our customers don’t know what the team should build. What should we do?

Allies
Purpose
Whole Team
Context

Do you have a clear, compelling purpose? If so, your on-site customers should know where to start. If not, your team is missing important customer skills. In this case, you can use traditional requirements gathering techniques, such as [Wiegers 1999], but you’re better off getting people with the skills you need. If you haven’t done so already, try chartering your team’s context and using that to better understand and advocate for the skills you need.

What if a customer review finds too many problems for us to deal with?

This is most likely to happen with a new team, before the team’s developers and customers have learned how to work together. In the short term, you’ll just need to write new stories for the changes needed.

Allies
Mob Programming
Pair Programming

Longer term, customers can spend more time each day talking with developers about their expectations and reviewing in-progress work. Mob programming is the ultimate expression of this idea. Customer/developer pairing is another option. With practice, you’ll learn to anticipate each others’ needs.

As a programmer, I’m offended by some of the things customers find in their reviews. They’re too nitpicky.

Things that can seem nitpicky to programmers—such as the color of the screen background, or a few pixels of alignment in the UI—represent polish and professionalism to customers. This goes both ways: some things that seem important to programmers, such as quality code and refactoring, often seem like unnecessary perfectionism to customers.

Rather than getting upset about these differences of perspective, try to learn what your customers care about and why. As you learn, you will anticipate your customers’ needs better, which will reduce the need to make changes.

Prerequisites

The team must include people who have the time and skill to work out requirements details.
Ally
Whole Team
No Bugs

In order to incrementally define requirements, the team must include people who have the time and skill to work out requirements details. Without them, your team will struggle with insufficient and unclear requirements.

When performing customer reviews, think of them as tools for conveying the customers’ perspective rather than as bug-hunting sessions. Developers should be able to produce code that’s nearly bug-free. The purpose of the review is to bring customers’ expectations and developers’ work into alignment.

Some organizations value written documents so highly that you can’t eliminate work-in-progress documents. Talk with management about why those documents are important and whether direct communication can replace them. Perhaps handoff documents are an acceptable compromise. If not, you don’t have to eliminate the documents, but you do need to include time for them in your stories.

Indicators

When customers work out requirements incrementally:

  • Developers are able to work on established stories while customers figure out the details for future stories.

  • Customers have ready answers to requirements questions, which allows planning and development to proceed quickly and smoothly.

  • By the time a story is done, it reflects customers’ expectations.

  • Customers experience no unpleasant surprises when reviewing software that’s been released.

Alternatives and Experiments

The traditional approach to requirements definition is to perform requirements analysis sessions and document the results. This requires an up-front requirements gathering phase, which takes extra time and introduces communication errors.

Incremental requirements is essentially taking that up-front requirements gathering phase and overlapping it with software development. Rather than customers writing a document, handing it to developers, and then developers reading it, customers speak directly to developers. This works because requirements details are communicated exactly when they are needed, so there’s little risk that they’ll be forgotten.

Ally
Mob Programming

As with stories, people usually experiment with this underlying idea by watering it down. Most often, their team doesn’t include people with customer skills, so they drift back towards a phase-based approach and handing off documents. That decreases agility. If you’re looking for opportunities to experiment, experiment in the other direction: increase communication, increase expertise, and decrease hand-offs. Mob programming is the result of one such experiment.

For handoff documentation, Alistair Cockburn suggests an intriguing experiment: rather than creating a written design overview, record a whiteboard conversation with an eloquent team member explaining the system to a programmer who isn’t familiar with it. Accompany the video with a table of contents that provides timestamps for each portion of the conversation.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Practice: Stories

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Stories

Audience
Whole Team

We plan our work in small, customer-centric pieces.

Stories may be the most misunderstood entity in all of Agile. They’re not requirements. They’re not use cases. They’re not even narratives. They’re much simpler than that.

Each story is a reminder to talk about something the team needs to do.

Stories are for planning. They’re the playing pieces of the planning game. That’s it! Alistair Cockburn calls them “promissory notes for future conversation.” Each story is a reminder to talk about something the team needs to do. They’re written on index cards, or virtual equivalent, so you can pick them up, move them around, and talk about how they fit into your plan.

Allies
Whole Team
Team Room
Incremental Requirements

Because stories are just a reminder to have a conversation, they don’t need to be detailed. In fact, detailed stories are a sign that people are missing the point, as the “Writers Workshop” sidebar illustrates. You’re supposed to have a whole team, a team room, and talk together regularly. The story is the starting point. The details come later, through Agile’s incremental, conversation-oriented approach to requirements.

Although stories are supposed to be brief, it’s okay to add additional notes when it’s helpful. If there’s something important you want to remember, or a technical detail that you need keep track of, go ahead and write it down. Just don’t feel obligated to add more detail. The card isn’t meant to be a requirements document. Just a reminder.

How to Create a Story

Ally
The Planning Game

Everything that your team needs to do, other than normal overhead, needs a story. That’s how work gets prioritized. No matter your role on the team, when you learn about something the team needs to do, write it on a card and queue it up for discussion in your next planning game. You just need to write enough to start the conversation and jog people’s memories in the future. For example:

  • “Warehouse inventory report”

  • “Full-screen demo option for job fair”

  • “TPS report for investor dog-and-pony show”

  • “Customizable branding on login screen”

You don’t have to use a template.

Some people like to use the Connextra template for writing stories, like this: “As a (role) I want (something) so that (result).” You’re welcome to use that format, if you want, but it’s not required. Connextra created that template as an experiment, to help remind them of how they were serving their users. You’re free to try your own experiments, or just jot down a few words.

Although stories are short, they still have two important characteristics:

  1. Stories represent customer value and are described in customers’ terminology. They produce an result customers value, not implementation details. This allows your on-site customers to prioritize them during the planning game.

  2. Stories have clear completion criteria. It doesn’t have to be written on the card, but on-site customers need to understand what it means for a story to be “done,” and be able to describe it when asked.

The following examples are not good stories:

  • “Automate integration build” doesn’t represent customer value.

  • “Deploy to staging server outside the firewall” describes implementation details rather than an end-result, doesn’t use customer terminology, and would likely be difficult for on-site customers to prioritize. “Make demo available to customers” would be better.

  • “Improve search performance” doesn’t have clear completion criteria. Similarly, “Make searches fast enough for my boss to be happy“ is clear, but still not specific. “Searches complete within one second“ is better.

Customer Value

Stories need to be customer-centric. Write them from customers’ point of view, and make sure they provide something that customers care about. (Remember that your “customers” include users and business stakeholders, not just your company’s literal customers.) On-site customers are in charge of priorities during the planning game, so if a story has no value to customers, they won’t—and shouldn’t—include it in the plan.

Customer-centric stories aren’t necessarily valuable to the end-user, but they should always be valuable from the perspective of on-site customers. For example, a story to produce a trade-show demo doesn’t help end-users, but it helps your business sell the product.

A good way to ensure that your stories are customer-centric is to ask your on-site customers to write the stories themselves.

One practical result of customer-centric stories is that you won’t have stories for technical issues. There should be no “Create build script” story, for example—customers wouldn’t know how to prioritize it. Although developers can tell customers where the technical stories belong in the plan, that disrupts the team’s balance of power and can leave customers feeling disenfranchised.

Allies
Evolutionary Design
Evolutionary Architecture

Instead, include any technical considerations when sizing each story. If a story requires that developers create or update a build script, for example, include that work in the story’s size. This is easier if you use evolutionary design and architecture.

Ruthless prioritization is the secret to shipping faster.

Developers often struggle with making stories customer-centric. Keep practicing! Without customer-centric stories, on-site customers have trouble making good prioritization decisions, and ruthless prioritization is the secret to shipping faster.

Developers, to make your stories customer-centric, talk them over with your on-site customers. Explain the outcome you’re trying to create and ask the customers to write the story in their own words. If you need to, you can add a note to remind you of the technology the story relates to.

For example, if you think that the team needs to switch to another database technology, you could explain it this way: “We need to move our articles to a database that handles full-text search better. Searches already take over half a second, and it will keep getting worse as more articles are added.“ The story might read, “Stop article search performance from getting worse. (full-text search database)”

Splitting and Combining Stories

Although stories can start at any size, they need to be “just right” by the end of the planning game. For most teams, “just right” means 4-10 stories per week, or about a day of work if the team combines their effort. To get there, you’ll need to combine small stories and split large ones.

Ally
Capacity

Combining stories is easy. Take several similar stories, staple them together, and write your new estimate on the front, if you’re using estimates. In a virtual team room, cut and paste the story descriptions onto a single virtual card.

Splitting stories is more difficult, because they need to remain customer-centric, and preferably releasable, as the “Keep Your Options Open” section describes. The trick is to find the essence of the story, not the individual steps it involves. What is the fundamental value the story provides, and what are the embellishments that build on that fundamental value? Write that fundamental essence as a story, and each embellishment as additional stories.

If that’s hard to imagine, Mike Cohn has an excellent chapter on how to split stories in his book Agile Estimating and Planning. [Cohn 2005] He provides the following options. (The examples are my own.)

  • Split according to priorities. For example, the check-out page of an online shopping site could be split into “check out“ (high priority), “coupons” (medium-high priority), and “gift receipts“ (medium-low priority), and “gift wrap“ (low priority).

  • Split along the boundaries of data supported by the story. For example, a story about collecting billing information could be split into “collect credit card info,” “collect gift card info,” and “collect PayPal info.”

  • Split based on the operations that are performed within the story. For example, a story about collecting credit card information could split into “collect credit card info” and “verify credit card info.”

  • Split into separate CRUD (Create, Read, Update, Delete) operations. For example, a story about administering customer data could be split into “list customers,” “sort customers,” “view customer”, “edit existing customer”, “add customer,” and “delete customer.”

  • Split out cross-cutting concerns (such as security, logging, error handling, and so on) by making multiple versions of your stories. For example, a story about charging credit cards could be split into “charge credit card,” “log credit card charges,” and “handle credit card rejection.”

  • Split out nonfunctional concerns (such as performance, stability, scalability, and so forth). For example, a story about charging credit cards could be split into “charge credit card“ and “support 100-200 credit card transactions per minute.”

Some of these techniques will lead to story clumps (stories that have to be released together); some won’t. It’s best to avoid story clumps, but don’t worry if you have trouble. You’ll get better with practice.

Special Stories

Most stories will add new capabilities to your software, but any work that requires the team’s time and isn’t part of their normal day-to-day work needs a story.

Documentation stories
Allies
Incremental Requirements
Evolutionary Design

Agile teams need very little documentation to do their work, due to their use of incremental requirements and evolutionary design, but you may need the team to produce documentation for other reasons. Stories for creating documentation are just like any other: make them customer-centric and be sure you can identify specific completion criteria. An example of a documentation story is “add billing feature to user manual.” See the “Documentation” section of the “Incremental Requirements” practice for details.

Bug stories
Allies
No Bugs
Done Done

Ideally, your team will fix bugs as soon as they find them, before declaring a story “done.” Nobody’s perfect, though, and you will miss some bugs. Track these bugs with stories such as “Fix multiple-user editing bug.” Schedule them as soon as possible to keep your code clean and reduce your need for bug-tracking software.

Bug stories can be difficult to size. Often, the biggest timesink when debugging is figuring out what’s wrong, and you usually won’t know how long that will take until you’ve done it. Instead, put a timebox around it: “We’ll spend up to a day investigating this bug. If we haven’t fixed it by then, we’ll discuss it as a team and schedule another story.” On the card, write “timebox 1 day.”

“Nonfunctional” stories

Performance, scalability, and stability—nonfunctional requirements—should be scheduled with stories, too. Like all stories, they need a concrete, customer-valued goal. Unlike other stories, though, you’re likely to need developers to help define the goal. It’s not enough to say, “the software needs to be stable” or “the software needs to be fast.” You need to be able to describe exactly how stable or fast it should be.

When writing nonfunctional stories, think about acceptable performance—the minimum necessary for satisfactory results—and best possible performance—the point at which further optimization adds little value.

Why have two numbers? Nonfunctional requirements, particularly performance optimization, can consume an infinite amount of time. Sometimes you reach your “best” goal early: this tells you when to stop. Other times you struggle even to meet the “acceptable” goal; this tells you when to keep going. For example, the performance criteria for a web page might be, “Support 500-1,000 requests per minute, with a latency of 50-200 ms.”

Like bug stories, nonfunctional stories can be difficult to size. You can use time-boxes for them, too.

Spike stories
The purpose of a spike story is to size another story.

Sometimes, developers won’t be able to size a story because they don’t know enough about the technology required to implement the story. When this happens, create a spike story to research that technology. The purpose of a spike story is to size the other story, not to research every nook and cranny. For example, “Figure out if ‘send HTML email’ story should be split.” Or, more concisely, "Spike ‘send HTML email.’”

Allies
Spike Solutions

They’re called spike stories because you’ll often use a spike solution to do the research. But you don’t have to.

Clean-up stories
Allies
Slack

Agile teams are expected to work in a way that allows them to continue indefinitely, and this includes having enough slack in their schedule to always leave their code better than they found it. If you do this, you shouldn’t need to explicitly schedule time for clean-up. The parts of the code you work on most often will automatically get cleaner over time.

If your code doesn’t get cleaner over time, something has gone wrong. Check out the “Slack” practice for ideas.

That said, sometimes it’s worth spending extra effort to clean up a particularly problematic piece of code. Like any other story, clean-up stories need to be customer-centric. They must be entirely optional; your code shouldn’t require clean-up stories to stay clean. You don’t ever want to be in a position where developers have to ask customers for permission to do their work, and you don’t want customers to have to ask developers for permission to ignore a story.

When you describe clean-up stories, speak in terms of the business benefit they will provide. Often, that benefit is in the form decreased development time. For example, rather than saying “refactor authentication service,” say “prevent login outages” or “decrease time required to make authentication changes.” You probably won’t be able to quantify the improvement, but be prepared to provide examples that illustrate the benefit. Like this: “Know how we’re having login outages every month or two? This will fix that.”

Allies
Refactoring

For clean-up stories don’t have a clear stopping point, use a timebox. To make sure you can stop when your time is up, avoid rewriting code. Instead, incrementally refactor it, keeping the code working at all times.

Meetings and overhead

Team activities such as the planning game, all-hands meetings, and other organizational overhead don’t require stories. If you have an unusual time commitment, such as training or an off-site day, you can reserve time for it with a story, but it’s often easier to absorb it like any other overhead.

Architecture, design, refactoring, and technical infrastructure
Allies
Evolutionary Design
Evolutionary Architecture

Other than clean-up stories, which usually shouldn’t be necessary, don’t create stories for technical details. Development tasks are part of the cost of implementing stories. Use evolutionary design and architecture to break large technical prerequisites into small pieces that you can implement incrementally.

Questions

Our customers understand development. Do we still have to write stories to be customer-centric?

It’s much more difficult to create customer-centric stories than developer-centric stories, so it’s tempting to find excuses to avoid them. “Our customers don’t mind if we have developer-centric stories” is one such excuse.

Even if customers understand developer-centric stories, customer-centric stories lead to better plans. Developer-centric stories usually focus on technical details, but your plan should be about value and releases.

If your customers are developers—if you’re developing software for developers, such as a library or framework—then your stories should use developer-centric language. Even so, they should reflect your customers’ perspective. Create stories about your customers’ needs, not your plans to fulfill their needs.

Shouldn’t developers make the software fast by default? Why do we need performance stories?

Nothing comes for free in software development. You don’t have to create performance stories (or other nonfunctional stories), but you do need to know what “fast” means in your situation, and developers will need to spend time to make it happen. Splitting nonfunctional requirements into separate stories doesn’t change the amount of time required; it just gives you more insight and control over how that time is spent.

Prerequisites

Ally
Incremental Requirements

Stories are no replacement for requirements. You need another way of getting details, whether through on-site customers and incremental requirements (the Agile way) or a requirements document (the traditional way).

Indicators

When you use stories well:

  • On-site customers understand all the work they approve and schedule.

  • Your team works on small, manageable pieces and makes customer-valued progress multiple times per week.

  • Stories are cheap to create and easy to discard.

Alternatives and Experiments

The main distinction between stories and the line items in most plans is that stories are customer-centric. If you can’t use customer-centric stories for some reason, customers cannot participate effectively in the planning game. This will eliminate one of its primary benefits: the ability to create better plans by blending information from both customers and programmers. You’ll also have trouble demonstrating progress to stakeholders. Unfortunately, no alternative practice will help.

Stories are such a simple idea, there isn’t a lot of opportunity for experiments. The most common change is to make them more complicated—typically, because people want to replace talking with documentation. I’ve yet to see those changes result in improvements.

Another common change is to record stories in a spreadsheet, or electronic planning tool, rather than on index cards. That can make lists of stories easier to read, but it makes collaboration more difficult. It’s a net loss that’s hard to appreciate without experience. Give cards at least three months before trying alternatives. If your team is remote, use virtual index cards in your free-form collaboration tool.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Practice: The Planning Game

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

The Planning Game

Audience
Whole Team

Our plans take advantage of both business and development expertise.

You may know what you want to release, but how do you actually construct a step-by-step plan? That’s where the planning game comes in.

Despite the name, the planning game isn’t a trivial piece of entertainment. In economics, a “game” refers to a situation where “players select actions and the payoff depends on the actions of all players.”1 That’s what the planning game is. It’s a cooperative game designed to create the best possible payoff.

1This definition of “game” comes from Deardoff’s Glossary of International Economics: http://www-personal.umich.edu/~alandear/glossary/g.html#game

The planning game is notable for the way it maximizes the amount of information contributed to your plan. It’s strikingly effective. Although the planning game has limitations, if you work within them, I know of no better way to create your detailed plan.

Allies
Adaptive Planning
Purpose
Visual Planning
Stories

The planning game is just one part of your overall planning process. It’s a way of breaking SIVs (see the “Shippable Increments of Value” section) into stories. To recap:

  1. Purpose provides the overall goal (the vision) and current direction (the mission).

  2. Visual Planning provides the high-level map and milestones for achieving the purpose.

  3. The Planning Game provides the detailed steps for reaching your next milestone.

Allies
Capacity
Forecasting

At the end of the planning game, you’ll have a detailed set of stories that will guide your upcoming work. If desired, you’ll be able use them to judge capacity and make forecasts.

How to Play

Every decision to do something is also a decision not to do something else.

The goal of the planning game is to focus the team’s effort on doing the things that provide the best return on the team’s time investment. After all, every decision to do something is also a decision not to do something else.

Because developers have the most information about costs—they’re most qualified to say how much work it will take to implement a story—they size the stories.

Because on-site customers have the most information about value—they’re most qualified to say what is important—they prioritize the stories.

Ally
Whole Team

Neither group creates the plan unilaterally. Instead, both groups together, each with their areas of expertise, and play the planning game:

  1. Customers decide the scope of the plan.

  2. The whole team brainstorms stories.

  3. Developers size the stories.

  4. Customers prioritize the stories and add them to the visual plan.

  5. Repeat until the plan is complete.

Team members that have both development and customer expertise can play both parts of the planning game, but it’s better if they stick to one role or the other. The benefits of the planning game come from resolving the natural tensions that arise between value and cost. It’s easy to fool yourself into ignoring those tensions when you play both roles.

1. Customers decide the scope of the plan
Ally
Visual Planning

Product managers, prepare for the planning game by choosing the highest priority SIVs from the visual plan. Choose just enough to fill out your story planning horizon (see the “How to Create Your Plan” section). For example, if you use the planning horizons shown in the “Planning Horizons” figure, you would choose about one month’s worth of SIVs.

Error on the side of too few SIVs. It’s easy to add more later.

Kick off the planning game by reviewing the team’s purpose, describing how the SIVs you’ve chosen fit into the overall plan, and why these SIVs are the most important to do first. Then explain the SIVs in more detail and answer any questions the team has.

2. The whole team brainstorms stories
Ally
Stories

Use simultaneous brainstorming (see the “Work Simultaneously” section) to come up with the stories needed to ship each SIV. The “Stories” practice describes how to create stories. Write each one on an index card or virtual equivalent.

You may have some upcoming commitments, requests, or activities that don’t correspond to your top SIVs, but need to be done anyway. Make stories for those, too. Anything that requires the team to do something, other than normal company overhead, needs a story.

3. Developers size the stories

Developers, review the story cards and sort them into several groups:

  • Stories that are just right

  • Stories that are too big

  • Stories that are too small

  • Stories that you can’t size because you don’t understand them

  • Stories that you can’t size because of technical unknowns

Allies
Capacity

The stories are “just right” when the whole team can finish four to ten per week, or about six on average. Over time, you’ll get an intuitive sense for when a story is “just right.” To start out, though, you might want to use one of the estimating techniques described in the “Capacity” practice.

You can choose any size to be “just right,” but 4-10 per week is a good starting point. Smaller than that, and you end up spending too much time creating, organizing, and tracking stories. Bigger than that, and you’ll have trouble making steady progress.

For stories that are too big, work with your customers to split them into smaller stories. The “Stories” practice describes how. For stories that are too small, combine them with other stories. If you’re working with physical index cards, you can literally staple them together. (You can create a cleaned-up version later, if you want.)

For stories you don’t understand, ask your customers to clarify. Sometimes, they’ll need to spend some time figuring it out. Continue on with the remaining stories while they do.

For stories that involve technical unknowns, make a spike story (see the “Spike Stories” section).

4. Customers prioritize the stories
Allies
Visual Planning

Customers, once developers okay stories as “just right,” add them to your visual plan in rough order of priority.

For stories that don’t have a corresponding SIV, add them wherever it makes the most sense to your team. Putting them among the other stories, according to priority, can help ensure they aren’t forgotten.

If you don’t have a visual plan, create a simple prioritized list. Identify one end of your conference table, release planning board, or virtual workspace as “first” and the other end as “last,” then arrange your stories in relative priority order. Create a single list—no ties allowed. This will force you to make important trade-off decisions.

Some stories won’t be worth adding. They’ll be unimportant or too far in the future. Go ahead and discard them. They won’t make sense later, and you’ll have better information in the future anyway. If you can’t stand to discard your stories, archive them somewhere out of sight.

Customers, make sure that you understand each story that you add. You need to make good decisions about what’s “in” and “out” of each release, as well as deciding the order in which stories will be built, and that means the stories need to match your perspective. Don’t let developers bully you into adding a story you don’t understand.

Developers, you have final say over whether stories are “just right” and ready to be prioritized. Don’t let customers bully you into accepting a story that’s too big or has too many unknowns.

5. Repeat until the plan is complete

Continue creating, sizing, and placing stories until you’ve filled out your story planning horizon. Bring in additional SIVs as necessary. For example, if you’re aiming for six stories per week and your story planning horizon is four weeks, continue until you have 24 stories. (Six stories per week multiplied by four weeks is 24.)

It’s often most convenient to completely flesh out each SIV, so you might end up with more stories than you technically need. For example, if your goal is to have 24 stories, and your two highest-priority SIVs had a total of 18 stories, you might bring in a third SIV and end up with 29 stories total.

Once you have all the stories you need, take a step back and double-check the plan:

  • Is every story sized “just right,” according to team members with development skills?

  • Does every story that couldn’t be sized have a spike story prioritized before it?

  • Have stories been added to the visual plan in the correct order, according to team members with customer skills?

  • Have irrelevant stories been discarded or archived out of sight?

  • In general, does the plan make sense and help the team accomplish its purpose?

Ask the team to consent to the plan (see the “Seek Consent” section). When they do, you’re done.

Keep Your Options Open

Try to create a plan that allows you to ship at any time. You don’t have to actually ship all the time, but you want to be able to ship at any time, even in the middle of working on a SIV.

By “ship,” I mean “available to be used.” Teams using continuous deployment deploy their software every day, but some improvements are turned off. They don’t count as “shipped” until they’re turned on. See the “Shippable Increments of Value” section for details.

Why do this? It allows you to keep your options open. Normally, if an exciting new opportunity comes along while you’re in the middle of building a SIV, you have to either wait until the SIV is done, throw away the partially done work, or set it aside to metaphorically rust. (See “Key Idea: Minimize Work in Progress”.) But if you plan your work so that you can ship at any time, you can decide that half of the SIV’s value is better than nothing, release what you have, and start working on the new opportunity immediately.

To keep your options open, build your plan so that each of its stories stands alone. You should be able to ship after every story. Subsequent stories can build on previous stories, but they shouldn’t depend on later stories.

For example, suppose you’re creating a checkout page that gets payment info from a user, validates it, and then charges their card. You might initially create a story for each layer of your architecture: “Get payment info on front-end,” “Store payment info on back end,” and “Process payment using third-party processor.” These are sometimes called horizontal stripes. It’s an easy way to create stories, but it prevents you from releasing your software until you finish all three stories. They form an all-or-nothing clump.

A better approach is to create stories that include all three horizontal layers but provide narrower individual utility. For example, you might create the stories “Process payment,” “Remember one card,” and “Remember and manage multiple cards.” These are vertical stripes (see the “Horizontal and Vertical Stripes” figure). Each builds on the previous, but you can ship after each one.

A diagram of a large box labelled “Application.” The box is split into three horizontal stripes and three vertical stripes. The horizontal stripes are labelled “front-end,” “back-end,” and “service integration.” The vertical stripes are labelled “Process payment,” “Remember one card,” and “Remember and manage multiple cards.”

Figure 1. Horizontal and vertical stripes

Don’t worry too much if you have trouble making your stories build on each other in this way. It takes practice. Being able to ship after each story gives you more flexibility, but a few story clumps in your plan won’t hurt much. With experience, you’ll learn to make your plans less lumpy.

How to Win the Planning Game

When developers and on-site customers come together to play the planning game, something amazing happens. I call it the miracle of collaboration. It’s a miracle because time appears out of nowhere.

As you can imagine, this miracle isn’t easy to achieve. When developers say a story needs to be split, customers often ask a question that causes developers’ teeth to grind: “Why does it cost so much?”

The instinctive reaction to this question is defensive: “It costs so much because software development is hard! Why are you questioning me!?”

Developers, there’s a better way to react. Reword the question in your head: “Help me understand my options.” Answer by talking about what’s easy and what’s difficult.

For example, imagine that you’re building a toaster, and your product manager has a story to automatically pop up the toast when it’s done. Developers say the story needs to be split, and when the product manager asks why, the developers calmly answer, “Well, popping up the toast is easy; that’s just cutting power to the electromagnet. But detecting when the toast is done—that’s new. We’ll need an image sensor and machine learning to accurately detect changes in brownness across all types of bread. Marbled rye... that’s gonna be tricky. Not to mention toaster pastries!”

That gives the product manager an opportunity to ask, “What about all those other toasters out there? How do they know when the toast is done?”

The developers make a face. “Oh, that’s a total kludge. They don’t detect when the toast is done at all. They just use a timer.”

Now the product manager can reply, “That’s okay! Our customers don’t want a super toaster. They just want a regular toaster. Use a timer like everyone else.”

“Oh, okay. Well, that won’t be hard at all. We don’t need to split this story after all.”

In general, customers tend not to know what’s easy, and they end up creating stories that are difficult to implement. Similarly, developers tend not to know what customers think is important, and they end up creating stories that aren’t valuable.

With open and honest communication, these conflicting tendencies can be reconciled. When customers ask for something unimportant but difficult, developers point out the expense and offer easier alternatives. Customers change direction. Time appears out of nowhere. It’s the miracle of collaboration.

Prioritizing Development Decisions

Customers want to ship a solid, usable product. They have to balance that desire with the desire to save money and meet market windows. As a result, they sometimes ask developers to skip important technical work. They do so because they aren’t aware of the nuances of development trade-offs in the same way that developers are.

If a development decision isn’t optional, it isn’t a story.

Developers, you are the most qualified to make decisions on development issues, just as customers are most qualified to make decisions on business issues. If a development decision isn’t optional, it isn’t a story. Don’t ask customers to prioritize it. Just do it. Either don’t mention the work at all—it’s a detail—or mention it as part of the cost of doing business:

We need to create an automated build when we implement our first story. So we need the very first story to be something tiny, such as just showing the page title.

When there is a business choice to be made, don’t ask customers to choose between technical options. Instead, interpret the technology and describe the options in terms of business impact.

To revisit the toaster example, rather than describing the choice of optical sensor versus timer like this:

We’re thinking about using a Mark 4 Wizzle-Frobitz optical sensor here for optimal release detection. Our other option is to use a 555-compatible countdown IC. The optical sensor is better, but we’d have to train a custom ML model, which is nice, because we’ve been wanting to dig into TensorDough. It’s very sophisticated, cutting-edge stuff that will allow us to detect the exact degree of brownness of the bread. The 555 is ancient tech and won’t give us dynamic detection abilities, but it won’t need any extra stories. Which would you prefer?

Try this instead:

We have two choices for deciding when toast is done. We can use a camera or a timer. The camera will allow us to toast the bread to the user’s exact preference, but will require several more stories. The timer won’t take any extra work, but the user is more likely to have undercooked or burned toast. Which would you prefer?

For more ideas about how to phrase stories in terms of business priorities, see the “Stories” practice.

Facing Reality

Customers, the planning game is almost certain to give you information that makes you unhappy. Even if your team isn’t making estimates or forecasts, the planning game will give you a rough idea of how much work there is to do. And, almost always, it will be more than you were hoping for.

Ignoring unpleasant reality won’t make the work take less time.

You may feel tempted to blame the messenger, and stop playing the planning game. Or you might pressure developers to not split stories. That would be a mistake. Ignoring unpleasant reality won’t make the work take less time; it just means you’ll be blindsided by delays. To paraphrase David Schmaltz: every release has a certain amount of disappointment associated with it. You can either use the planning game to dole out the disappointment in measured doses... or you can save it all up for the end.

Some people worry that developers will use the planning game to pad their work and slack off. That fear is misguided. In my experience, developers are highly-educated professionals who are motivated to meet customer expectations. Steve McConnell agrees: “Software developers like to work. The best way to motivate developers is to provide an environment that makes it easy for them to focus on what they like doing most, which is developing software... [Developers] have high achievement motivation: they will work to the objectives you specify, but you have to tell them what those objectives are.” [McConnell 1996] (pp. 255-256)

Although your plan may be bigger than you like, if developers are asking you to split stories, it’s most likely because they want to set realistic expectations. In practice, I find that developers don’t split stories enough, at first, rather than the other way around. But if they do make their stories too small, it just means they’ll get more stories done, and you can compensate in your next planning game.

Iterating the Planning Game

Allies
Visual Planning

As your team finishes stories, remove them from the plan. When it shrinks below your story planning horizon—for example, less than 24 stories—it’s time to play the planning game again and make more stories. Afterwards, double-check if you need to pull more SIVs into your visual plan.

Stakeholders will also suggest new ideas and features. Some won’t be worth doing, and can be discarded. (Politely!) Others will be good ideas for future SIVs, and can be added to the less-detailed parts of your visual plan. Ideas you want to do soon, though, need stories. Those should be brought to the team to be sized and prioritized.

With practice, it should only take a few minutes to discuss, size, and prioritize a story. Accordingly, some teams review new stories the day they come in. Other teams schedule bigger planning game sessions every week or two. I find small, frequent sessions to be less tiring than big, less frequent sessions, but both approaches work.

Sometimes stakeholders will try to end-run the prioritization process by talking to programmers directly. The correct response is to listen to their request, write it down—I carry index cards with me for this exact purpose—and tell them it will be prioritized the next time your team meets to plan. Then you can hand it off to a team member with customer skills to follow up.

Questions

How can we encourage stakeholders to use stories to request changes?

It’s not necessary. Instead, team members with customer skills should interpret stakeholders’ ideas and turn them into stories for the team.

What do we do about technical infrastructure and prerequisites?

Allies
Evolutionary Design
Evolutionary Architecture

Agile planning starts from the assumption that you can break your work into small, customer-centric stories, and build your technical infrastructure as you go. Delivering zone teams use evolutionary design and architecture for this purpose.

If you aren’t using evolutionary design and architecture, then try to think of ways to split your technical infrastructure into small pieces that you can implement as part of your other stories. If you can’t think of any, you’ll need to make an infrastructure story, or architecture story, and tell your on-site customers that it’s a prerequisite for certain stories. Try to avoid this, as it takes away their agency and makes your planning much more complicated.

[Denne and Cleland-Huang 2004] describes an approach to planning that allows you to incorporate technical prerequisites. Evolutionary design is a better choice, if you can use it.

How do we make time for paying off technical debt?

Allies
Slack

“Technical debt” is commonly used to refer to code that needs extra work to clean up. In general, you should have enough slack to clean up everything you touch, as part of every single story, so that you always leave your code better than you found it.

Sometimes, though, you’re dealing with code that needs extra attention. In that case, you can use clean-up stories. See the “Clean-Up Stories” section.

Prerequisites

Allies
Whole Team
Evolutionary Design
Evolutionary Architecture

The planning game relies on several simplifying assumptions:

  • Team members with customer skills who can make wise prioritization decisions.

  • Team members with development skills you can reliably size stories.

  • Customer-centric stories with minimal dependencies on each other.

The final point requires that technical infrastructure be built incrementally, as part of your customer-centric stories. If you have trouble with this, you’ll need Delivering fluency, or at least evolutionary design and architecture.

Indicators

When you play the planning game well:

  • Customers and developers all feel they have contributed to the plan.

  • Feelings of pressure and stress are focused on the constraints of the plan, and possible options, rather than on individuals or groups.

  • Developers suggest options for reducing work while still fulfilling the team’s purpose.

  • Customers ruthlessly prioritize the stories that best serve the team’s purpose.

Alternatives and Experiments

The key idea of the planning game is that customers and developers come together to make a plan that’s better than either could make on their own. It’s a team-based approach, not an individual-based approach, and it’s focused on results, not tasks.

I haven’t seen anything that beats this core idea, but there’s room to experiment. For example, some teams prefer to create very small stories that can be finished in an hour or two. (Be sure to keep them customer-centric, though! Otherwise, they’re just tasks.) You can also experiment with the planning game agenda. For example, you could try creating and sizing your stories asynchronously, including more or fewer people, or planning more or less frequently.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

Jade Rubick on Single-Threaded Owners

Jade Rubick has an interesting experience report about the Single-Threaded Owner model of organization on his blog.

I often saw issues where the engineering manager (EM) and product manager (PM) didn’t work well together, and these teams tended to do quite poorly. Generally what would happen is their managers would attempt to “coach them through it”, but what would happen in practice is that those teams would just not work effectively for six months or more.

The Single-Threaded Owner (STO) model resolves these problems by creating cross-functional teams, composed of engineering, product, and design, with a single leader.

My favorite part is the story of the successful STO team. It sounds a lot like what I see from fluent Optimizing zone teams:

They were more strategically aligned than most any local team I’ve worked with...

They took greater ownership of their domain and began challenging some of the product direction they were “given”. They successfully pivoted the team’s direction to something they thought more important...

The team became incredibly productive and incremental, to the point that they started delivering things to customers every couple of weeks...

They would often learn new information from customers and make a small 1-2 week project that would start the next week...

It wasn’t all smiles and roses, though. Jade ran into substantial resistance. The whole experience report is worth reading. Check it out here.

AoAD2 Practice: Visual Planning

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Visual Planning

Audience
Product Managers, Customers

We have a map for achieving our purpose.

Your plans are the key to achieving your team’s purpose. Rather than building your plan as a dry, dead list, create a plan that lives and breathes. Rather than showing what to do, show options, relationships, and create the ability to make choices. Strike off the shackles of simple prioritization and let your plans fly free.

A visual plan allows you to visualize your choices and adapt your plans.

More prosaically, visual planning is a way of mapping out your options. Rather than saying “do this, then this, then that,” a visual plan allows you to visualize your choices and adapt your plans as you go.

Ally
Real Customer Involvement

Visual planning is led by the team’s product manager, with the assistance of the team’s other on-site customers. Do your best to include key stakeholders, too, at least for high-level planning. Their perspective will improve the quality of your plans.

Developers can be heavily involved, or not, as your team sees fit. Some developers prefer not to attend yet another meeting—and, in truth, it’s possible that their time could be better spent elsewhere. On the other hand, developers’ work will benefit from an in-depth understanding of the plan, and their perspective often leads to better plans. I tend to leave it up to each individual developer to decide. When in doubt, include them.

Some product managers prefer to create the first draft of their plan by themselves, or with just one or two other participants. That’s fine, so long as you eventually include broader perspectives. Developer and stakeholder feedback will help you make sure nothing important is missed.

Ally
The Planning Game

Even if developers don’t attend, they still need to understand the plan and provide feedback. Be sure to set aside time to discuss the plan with developers. The planning game might be a good time to do so.

How to Create a Visual Plan

The right visualization is the one that works for your team and its stakeholders.

The possibilities for visual plans are endless. Here, I discuss four techniques. You follow one of these techniques as written, mix and match between them, or create new visualizations that are wholly your own. The right visualization is the one the works for your team and its stakeholders.

Cluster mapping

Cluster mapping is the simplest and most flexible way to visualize your plans, and also one of the most effective. It’s my go-to technique.

A diagram of a whiteboard. The board is labelled, “release plan.” A line divides the board into left and right halves. The left half is labelled, “Next release Aug 1.” It has three SIV cards, numbered 1, 2, and 3. Each SIV has 5-10 story cards organized in a cluster beneath it. The right half of the board is labelled, “Future releases.” It has two more clusters of a SIV with story cards (similar to the left hand side). The SIVs are numbered 4 and 5. It also another SIV with just one story, without a number, and then three clusters of only SIV cards. One of them consists of two cards and is labelled, “pick one.”

Figure 1. A cluster map

Allies
Purpose
Team Room

To create a cluster map, use simultaneous brainstorming (see the “Work Simultaneously” section) to generate SIVs and stories that will help your team fulfill its purpose. (Remember, SIVs are shippable increments of value, and stories are smaller pieces that may not be shippable on their own. See the “Shippable Increments of Value” section and the “Divide SIVs Into Stories” section.)

Focus more on SIVs than stories. You’re starting with a high-level map, not a detailed plan. The important thing is to think about your team’s purpose and what could help achieve it. But sometimes it’s easier to brainstorm stories and later combine them together into SIVs, so feel free to create both.

Be brief. You can always add more detail later.

For each SIV and story, write a brief phrase on an index card or its virtual equivalent, such as “TPS Report” or “Allow subscribers to change subscription tier.” Don’t worry about the phrasing. You just need to write enough to act as a reminder. You can always add more detail later.

If you have a virtual team room, use your free-form collaboration tool. If you have a physical team room, put the cards on your conference table or release planning board.

I like to color-code my cards. (I use red for SIVs and blue for stories.) For this exercise, though, don’t stress if a SIV turns out to be a story, or vice-versa. You can fix the colors later.

Once you’re done brainstorming, use affinity mapping (see the “Work Simultaneously” section) to cluster the SIVs and stories. As you work, you’ll probably come up with more ideas. Go ahead and add them.

Some of your cards will be duplicates, and you can either discard or combine them. When using physical index cards, the easiest way to combine cards is to literally staple them together. You can create cleaned-up versions later, if you want.

If you brainstormed a lot of stories, they’re likely to form clusters of their own. Create a SIV for each cluster. You can either discard the stories—you’ll have a chance to make better stories during the planning game—or you can keep them grouped together with their SIV. With physical cards, sometimes it’s useful to use a rubber band or binder clip to attach the stories to their associated SIV.

When you’ve finished your affinity map, transfer it to your release planning board, if it isn’t already there. Then take a step back and review. Tweak the clusters and add notes that make the map easier to explain. You want to be able to use the map as a visual aid to describe how your team is going to achieve its purpose.

Some cards, or even clusters, could represent different ways of accomplishing the same thing. If you’re not sure which choice you want to make, leave the both on the board! Your plan is a map, and maps show multiple routes to a destination. You might even want to make some experiment-style SIVs to help you choose between options.

Some clusters or cards may turn out to be irrelevant to your current purpose, or too far in the future. Go ahead and discard them for now. If you can’t stand to do that, archive them somewhere out of sight. Keep your map focused on what matters.

Speaking of purpose, you may have some work to do that isn’t directly related to your team’s purpose, but has to be done anyway. If you haven’t made cards for those needs, go ahead and do so, and add them to the board.

At this point, you have a complete high-level plan showing the SIVs that your team could work on. This is a good time to take a break and get feedback from team members and stakeholders who didn’t attend. Next, you’ll go into more detail.

Breaking down SIVs
Ally
Adaptive Planning

As described in the “Adaptive Planning” practice, your plan will consist of multiple levels of detail (see the “How to Create Your Plan” section). Your visual plan involves these levels:

  1. Purpose

  2. High-level SIVs

  3. Smallest possible SIVs

  4. Stories

You started with your team’s purpose and used that to create high-level SIVs. Now you’ll break them down further. Starting with your most valuable SIVs, brainstorm smaller SIVs that are still shippable and valuable. For example, if your product is an online store and you have an “enhanced checkout page” SIV, you could break it down into “remember credit cards,” “check out with PayPal,” “gift wrapping,“ “coupons,“ and so forth.

Sometimes it’s helpful to brainstorm the stories associated with each SIV, then move them around to form clusters representing smaller SIVs. If that doesn’t help, don’t worry; you can always split your SIVs later.

Keep going until you’ve broken down enough SIVs to fill your “small SIV” planning horizon. For example, if you use the planning horizons shown in the “Planning Horizons” figure, you would stop when you had about three months worth of small SIVs.

Just use your gut feel about when to stop. You don’t need to get a proper development estimate: you’re not committing to anything and there’s no harm if you get it wrong. If you don’t break down enough SIVs, you’ll just do more later; if you break down too many, you’ll just have done more planning than you needed to.

When your SIVs are as small as you can make them, prioritize them by marking each one with a number. (No ties allowed!) If you’re using physical index cards, put the number on a small sticky note and stick it to the card. As your priorities change, you can move the sticky notes without rewriting the cards.

That finishes off the “small SIVs” level of detail. This is another good time to take a break and get feedback.

Ally
The Planning Game

The final level of detail for your visual plan is to break your small SIVs into stories. Use the planning game to do so. When you’re done, you’ll have a set of stories for each of your highest-priority SIVs. Add each set to the board near its corresponding SIV. The final result will look something like the “A Cluster Map” figure.

Impact mapping

Sometimes you want more structure than cluster mapping provides. Goyko Adzic’s Impact Mapping [Adzic 2012] is a great tool for helping you explore your options.

Find the shortest path through the map to your goal.

Never aim to implement the whole map. Instead, find the shortest path through the map to the goal! [Adzic 2012] (pp. 12-13)

Gojko Adzic

A diagram of a whiteboard. The board is labelled, “release plan.” In the center of the board is a circle with the text “$200K MRR, >20 NPS.” The circle is labelled “why (goal).” Several levels of cards (shown as simple rectangles) extend from the center circle in the manner of a mind map. Starting with the first, the levels are labelled “who (actors),” “how (impacts),” “what (deliverables),” and “small SIVs.” Some of the “how“ cards are covered in dots; they’re labelled “dot votes.” The cards with a lot of dots are the only ones that have “small SIV” cards. Five of the small SIVs are prioritized 1-5, and the first three have a set of cards labelled “stories” next to them.

Figure 2. An impact map

An impact map is a type of mind map. In case you aren’t familiar with them, mind maps are hierarchical trees of ideas. You start with the core idea in the center of the map. Ideas related to that core idea branch out from the center to create their own nodes. For each of those ideas, additional ideas branch out. And so forth.

In the case of an impact map, “Why” (goal) goes in the center, followed by “Who” (actors), “How” (impacts) and “What” (deliverables).

To create the impact map, use your release planning whiteboard (if in person) or a collaborative mind mapping tool (if remote). You can use your free-form collaboration tool if you don’t have a mind mapping tool. If you’re using a physical board, you might want to use index cards for the nodes. That will make it easier to move them around as needed. As always, work simultaneously rather than bottle-necking contributions behind a single person.

Ally
Purpose

Start with your goal. This is the “Why” of the impact map, and it goes in the center. Your goal should relate to your team’s purpose in some way. It might be a condensed version of your mission, the next relevant mission tests, or both. In an impact map, everything stems from the goal. This is the destination your map will show you how to reach. To use the Team Sasquatch example (see the “An Example Purpose” sidebar), your goal might be “$200K MRR, >20 NPS.”

Ally
Context

Put actors that relate to the goal as the next level of your map. This is the “Who” level, and it includes everyone, other than your team, that can either help or hinder your team’s ability to reach the goal. If you’ve created a context diagram (see the “Boundaries and Interactions” section), the stakeholder groups in that diagram are a good starting point. Be sure to include actors with potentially negative impacts, such as competitors. Examples include “customers,” “prospects,” “sales and marketing,” and “regulators.”

Next comes impacts: the “How” of the impact map. Think of they way each actor could affect your goal. How could they help? How could they get in the way? How do you want the actor’s behavior to change? For example, existing customers could “recommend us on social media" or industry journals could “post positive reviews.” Be sure to include negative impacts, such as regulators who “increase auditing requirements“ or competitors who “change pricing model.” Talk about how the behavior is different than today; if regulators already require auditing, the impact isn’t just “require audits,” it’s “increase auditing requirements.”

Up until now, you’ve been thinking broadly and generating options. Now it’s time to focus your thinking. Which impacts are critical for meeting your goal? Which represent low-hanging fruit? Which represent assumptions that need to be tested? Choose the impacts that are the highest priority. In a group setting, dot voting can help (see the “Work Simultaneously” section).

The purpose of the impact map is to focus your attention on your goal and impacts.

Now you’re ready for deliverables, or the “What” of the impact map. These are your high-level SIVs. It’s tempting to think of deliverables as the most important part of the map, but they’re not! The purpose of the map is to focus your attention on your goal and the impacts you want to achieve (or mitigate). SIVs are what you use to do so, but it’s like saying your car is the most important part of a road atlas. Necessary for the trip, yes. The map—no.

For each impact, think of ways your team could support the impact (for positive impacts) or mitigate the impact (for negative impacts), or learn more about the impact (for assumptions that need testing). Keep your ideas high level. For example, to support customers recommending you on social media, you could add “Automatically post screen shots“ and “Post celebrations.” Later, you’ll create small SIVs such as “Post screen shot to Twitter” or “Facebook post celebrating new record.”

At this point, you have a high-level plan. Next, you’ll break the high-level SIVs down into small SIVs, and those further down into stories, using the approach described in the “Breaking Down SIVs” section. Add each level of detail onto your impact map: small SIVs attach to their corresponding “What“ node, and stories attach to their corresponding small SIV. The final result will look something like the “An Impact Map” figure.

Prospective analysis

Prospective analysis helps you generate ideas by imagining future outcomes. It’s particularly good as a risk management tool. You can use the prospective analysis as a planning tool on its own or as a lead-in to another planning approach.

A diagram of a whiteboard. The board is labelled, “release plan.” In the center of the board is a circle with the text “$200K MRR, >20 NPS.” The circle is labelled “why (goal).” Several levels of cards (shown as simple rectangles) extend from the center circle in the manner of a mind map. Starting with the first, the levels are labelled “who (actors),” “how (impacts),” “what (deliverables),” and “small SIVs.” Some of the “how“ cards are covered in dots; they’re labelled “dot votes.” The cards with a lot of dots are the only ones that have “small SIV” cards. Five of the small SIVs are prioritized 1-5, and the first three have a set of cards labelled “stories” next to them.

Figure 3. A prospective analysis

One type of prospective analysis is an Impact and Probability chart, described in Diana Larsen and Ainsley Nies’ book Liftoff: Start and Sustain Successful Teams [Larsen and Nies 2016]. It’s simple and effective.

To create the chart, draw a large graph on your release planning board or virtual equivalent. Label the vertical axis from -3 to +3 and draw a horizontal dashed line at the “0” mark. For the horizontal axis, label a range from “Won’t happen” to “50/50” to “Will happen.” Draw a vertical dashed line at the “50/50” mark.

Now use simultaneous brainstorming to think of what might happen in your team’s future—to the team, their stakeholders, and their software. Write each one on an index card. Be sure to think of positive results as well as negative.

Participants can add their cards to the chart immediately, or wait until the brainstorming is done. When you add them to the chart, position them according to the likelihood of the card happening (the horizontal axis) as well as the impact of the card happening (the vertical axis). The impact can range from “very bad“ (the “-3” mark) to “neutral“ (“0”) to “very positive“ (“+3”).

After the cards are on the board, take a moment to review and adjust their positions. This can be done simultaneously, similarly to affinity mapping or mute mapping. (See the “Work Simultaneously” section.) The result will look something like the “A Prospective Analysis” figure.

Next, use dot voting to choose the cards that are most important to address. You can use these priorities as input into one of the other visualizations. If you’re using the prospective analysis as a stand-alone plan, brainstorm high-level SIVs that will help achieve positive outcomes and mitigate negative outcomes. Add them next to the chart with arrows pointing to their corresponding cards.

Finally, break the high-level SIVs down into small SIVs and stories as described in the “Breaking Down SIVs” section.

Story mapping

Jeff Patton’s story maps are particularly useful when you’re building out new user interfaces. It provides a map for building user interaction flows incrementally. Story maps can be used on their own, or used to flesh out a subset of a larger plan created with the one of the other approaches.

A diagram showing a grid of cards (shown as simple rectangles). The top two rows of the grid are labelled “user activities” and “user tasks.” There’s a horizontal line below the second row. Each user activity card is associated with several user task cards, and the tasks are divided into six columns. The columns are labelled “SIVs.” The first three columns have additional cards and the last three columns are empty. In the first three columns, vertical rows of cards are grouped into several boxes, labelled “small SIVs.” Five of the boxes are labelled with numbers from one to five.

Figure 4. A story map

Allies
Purpose
Context

To create a story map, start by reviewing your purpose and context. Who are the users that will use your software? If they’re not the people paying for it, who are the choosers who will pay for your software? What benefits do they each get? How does that relate to your team’s purpose and the value your organization expects?

Next, map out the big picture. Starting with the most important users, brainstorm user activities—things they want to to use your software to do. Other systems can also be users. As always, you can brainstorm simultaneously (see the “Work Simultaneously” section). Patton gives the example of an email application, with activities such as “managing email,” “configure email servers,” and “set up out of office responses.” Write each one on an index card, sticky note, or virtual equivalent. A short phrase is enough.

Story maps can take up a lot of space. If you have a physical team room, you may want to use a large wall and sticky notes rather than a whiteboard and index cards. “Super sticky” sticky notes are best for longevity—you don’t want to come in after a long weekend and find your plan on the floor!

For each activity, brainstorm user tasks that users will perform to accomplish their activities. For the “managing email” activity, Patton gives the examples of “send message,” “read message,” “delete message,” “mark message as spam,” and so forth. Arrange them in the order that the tasks will be performed, and put them under each activity in a horizontal line.

As you work, tasks will inspire more activities and users, which will in turn inspire new tasks. Keep going until you have a broad overview of the ways people and systems will use your software, organized with the most important activities and tasks first.

Validate the map by “walking the board“ from start to finish, like this: “Our first activity is ‘managing email.’ Users will send messages, read messages, delete messages...” Now may be a good time to take a break and get feedback from developers, users, and other stakeholders.

Next, you’ll go into more detail. Starting with your most important activities and tasks, identify sets of cards that are independently shippable. Draw vertical lines to group them into high-level SIVs. Choose several to ship first, according to your “small SIVs” planning horizon.

Think broadly and come up with a variety of options.

For the SIVs that you’ve chosen, brainstorm subtasks and user interface details for each user task. Write them on cards, sticky notes, or their virtual equivalent, and arrange them vertically under their task according to priority. For example, for the “send message“ task, you might create cards such as “compose plain text message,” “compose HTML message,” “send via POP,” “send via SMTP,” “embed image,” “add attachments,” and so forth. Use this opportunity to think broadly and come up with a variety of options, without worrying about what’s in scope.

Now you can break your SIVs down into the smallest possible SIVs. For each SIV, choose sets of subtasks that can be shipped independently. Draw a horizontal line and move the remaining subtasks below the line. Repeat until each vertical column is divided into boxes. Each box represents a small SIV. Number each box in order of priority.

Ally
The Planning Game

Your next step will be to use the planning game to refine your stories. This is another good opportunity to take a break and get feedback. When you’re ready for the planning game, use the cards in each box as your initial stories. When you’re done, put the final set of stories back into your story map. The final result will look something like the “A Story Map” figure.

Iterating the Visual Plan

Update and improve your plan frequently.

Whichever visualization you choose, update and improve it frequently. The product managers I’ve known are constantly looking at and tweaking their plans. One put his cluster map in a shoebox and carried it with him everywhere. (He used binder clips to group the cards into clusters.) He was constantly spreading it out in meetings and making little changes.

At the very least, check in on your plan every week. Every time the team finishes a story or SIV, check your planning horizons and pull out more detail, as described in the “How to Create Your Plan” section.

Questions

What if we have to use a corporate planning tool that doesn’t allow us to make our own visual plans?

Don’t let your tools prevent you from making and customizing your visual plan. If you’re required to use a restrictive tool, treat it as another view into your plan. Use the visual plan as your source of truth and copy information into the corporate tool as needed. It’s wasteful to duplicate effort, of course, but a good visual plan is so valuable, it’s worth it. Over time, you may be able to convince management to relax the requirement. (See the “Delegate Authority and Responsibility to Teams” section.)

How do we convert our visual plan into a format that our organization and stakeholders want to see?

Ally
Roadmaps

There are a variety of options depending on how much detail your organization needs. See the “Roadmaps” practice for details.

Prerequisites

Ally
Team Room

Visual planning requires a team room designed for collaboration. For a virtual team room, this requires a free-form collaboration tool. For a physical team room, it requires a large table, magnetic whiteboard, and index cards or sticky notes.

Don’t use a dedicated planning tool. You need the ability to visualize, customize, and experiment.

It’s tempting, but don’t use electronic tools for your plan. If you’re a virtual team, you’ll have to use a free-form collaboration tool, but even then, don’t use a dedicated planning tool. They’re too restrictive, and most are limited to simple priority lists. You need the ability to visualize, customize, and experiment.

So-called Agile planning tools, also known as “Agile Lifecycle Management” tools, are anything but Agile. They’re built for the whims of big Cargo Cult Agile companies. They will harm your agility. Avoid them.

Ally
Informative Workspace

For in-person teams, physical maps are dramatically more effective than electronic maps, even with a free-form tool. The physical map forms an important part of your informative workspace. Their tactile, large-scale nature connects with our brains in a way screens just can’t. It’s hard to appreciate how much of a difference this makes until you’ve experienced it. Make an extra effort to use physical plans.

Allies
Whole Team
Purpose

Visual planning also requires the leadership of people with product management and customer skills. Without their participation, you can still create a visual plan, but it may not be a good plan. A clear purpose is also essential.

Ally
Real Customer Involvement

For best results, include key stakeholders in your planning sessions, or at least solicit their feedback on your draft plans. Without stakeholder feedback, you run the risk of tunnel vision. You could build something that seems right to you, but doesn’t actually fulfill stakeholder needs.

Indicators

When you create and communicate a visual plan well:

  • Stakeholders and team members not only understand what will be done, they understand why it will be done.

  • The relationships between the parts of your plan are easy to see.

  • Stakeholders and team members contribute new ideas as a result of their deeper understanding of context.

Alternatives and Experiments

You can start experimenting with this practice right away. Try each of the maps described here, mix and match, and include your own ideas. Find the approach that speaks best to you, your team, and your key stakeholders. There’s no right answer, so experiment with as many ideas as you can find or think of. Don’t be afraid to change things up, either; I change my visualizations every few months, and it always gives me new insights and fresh excitement for possibilities.

There are many ways of visualizing plans. One startup I worked with created a chart of business priorities in four categories (“business development,” “cost control,” “risk mitigation,” and “improving capacity.”) Each idea got an sticky note that was ordered by priority. There were hundreds of ideas—far more than the team could ever accomplish—and only a few were labelled with as top priorities. The rest were constantly moving around as the startup founders came up with new ideas and reconsidered old ones.

Another team had a lot of small date-based commitments. They created a commitment calendar with a column for each week. Cards representing the commitments due each week were put in the corresponding column. Every week, the team reviewed upcoming commitments and included them in that week’s task planning.

There are many more options available. Experiment freely! With one exception: Wait at least three months before using a dedicated planning tool. Even the better ones are terribly constraining, but it’s hard to see why that’s a problem until you’ve experienced the power and flexibility of a free-form approach.

Further Reading

Impact Mapping [Adzic 2012] is the definitive guide to impact maps. It’s a short, easy read, and chock-full of useful advice. If you’re using impact maps, it’s well worth getting.

Liftoff: Start and Sustain Agile Teams [Larsen and Nies 2016] talks about prospective analysis on pp. 95-97. It doesn’t say much than I do here, but it has a lot of other good material about chartering and facilitation.

XXX Jeff Patton Story Mapping book TBD

XXX Check out Luke Hohmann’s Innovation Games

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Practice: Adaptive Planning

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Adaptive Planning

Audience
Product Managers, Customers

We plan for success.

Imagine you’ve been freed from the shackles of predetermined plans. “Maximize our return on investment,” your boss says. “We’ve already talked about the purpose of this team. I’m counting on you to work out the details. Create your own plans and set your own release dates—just make sure we get a good return on our investment.”

Now what?

Shippable Increments of Value

Shippable Increments of Value (SIVs) are the building blocks of your plan.1 Each one represents concrete, valuable progress. SIVs have three parts:

1In the first edition of this book, I used the term “Minimum Marketable Feature” (MMF) instead of SIV. The term came from [Denne and Cleland-Huang 2004]. SIVs are the same idea, but I’ve changed the name because not everything that’s valuable is marketable.

  1. Shippable. When you finish working on the SIV, you can release it and reap its benefits.

  2. Increment. The SIV doesn’t try to do everything. It’s just one step in the right direction.

  3. Of Value. The SIV brings value to your organization in some way.

As the “What Do Organizations Value” sidebar discusses, there are many ways to provide value, from improving revenue, to reducing risk, to gathering information. Your SIVs will generally fall into these categories:

  • Features. Features are “shipped” when their target audience can use them. For example, if you create a new report for customers, the feature has shipped when real customers are able to run the report.

  • Experiments. Experiments are “shipped” when you’ve created the ability to make an informed decision. For example, if you want to decide whether to change your sign-up flow, you might create an A/B test.2 The experiment has shipped when the A/B test is active, the point at which the data will be evaluated has been decided, and the criteria for keeping or discarding the new flow has been determined.

  • Options. Options are “shipped” when you’ve gained the ability to postpone a decision. For example, rather than committing to a particular vendor, you might modify your software to also support a second vendor. The option has shipped when you can switch between the vendors at will.

2A/B testing is when you show different things to different groups of people and evaluate which one had the best results.

Experiments and options require people to be comfortable with uncertainty and ambiguity, so they tend to be used by Optimizing teams. But any team can use them.

Another way to think about SIVs is to ask yourself what would happen if the team shipped the SIV, then changed direction and never developed it further. Would it still provide value? If so, it’s a SIV. If not, it’s missing the value part.

For every SIV, you should be able to articulate three things:

Ally
Purpose
  1. Why it’s valuable

  2. How the value relates to the team’s purpose

  3. What “shipped” looks like, at a high level

Allies
Incremental Requirements
Visual Planning

The details come later. You’ll track your SIVs with a short phrase on a card in your visual plan, such as “TPS Report,” “sign-up flow A/B test,” or “authentication vendor independence.” If you want to keep more detailed notes, you can, but for most product managers I meet, a short phrase is enough of a reminder.

If you’re wondering how SIVs and other requirements are documented and communicated, see the “Incremental Requirements” practice.

Focus on One SIV at a Time

Stakeholders love it when teams work on multiple SIVs simultaneously. It feels like a lot of work is being done, and everything gets to be top priority! So easy. And so very, very wasteful. Focusing on one SIV at a time will improve your delivery speed and increase your value.

Consider a team that has three SIVs, as shown in the “Effects of Focusing On Value” figure. In this simplified example, each SIV has equal value: when complete, each will yield $4 in value every month. Each SIV takes two months to complete.

Four scenarios. Scenario A shows a team working on three SIVs simultaneously over six months. They earn $4 for each one in the seventh month for a total value of $12. Scenario B shows a team focusing on one SIV at a time. They work on SIV 1 in months 1-2 and it earns $4 in months 3-7. They work on SIV 2 in months 3-4 and it earns $4 in months 5-7. They work on SIV 3 in months 5-6 and it earns $4 in month 7. After month 7, SIV 1 has earned $20, SIV 2 has earned $12, and SIV 3 has earned $4, for a total of $36. Scenario C is similar to Scenario B, but it has six SIVs earning $2 per month. After month 7, SIV 1a has earned $12, SIV 1b has earned $10, SIV 2a has earned $8, SIV 2b has earned $6, SIV 3a has earned $4, and SIV 3b has earned $2, for a total of $42. Scenario D is the same as Scenario C, except SIVs 1a and 1b earn $3 per month, SIVs 2a and 2b earn $2 per month, and SIVs 3a and 3b earn $1 per month. After month 7, SIV 1a has earned $18, SIV 1b has earned $15, SIV 2a has earned $8, SIV 2b has earned $6, SIV 3a has earned $2, and SIV 3b has earned $1, for a total of $50.

Figure 1. Effects of focusing on value

In Scenario A, the team works on all three SIVs simultaneously. It takes them six months to finish all three. When they’re done, they start collecting $12 every month.

In Scenario B, the team works on one SIV at a time. They ship SIV #1 after two months—one-third the time of scenario A. It starts earning $4 every month while they switch to SIV #2. It ships after the fourth month—still faster than scenario A—and also starts earning $4 per month. Finally, after the sixth months, they finish SIV #3. They’ve earned $36 in the time it took Scenario A to earn $12. And that’s ignoring the costs of task switching and the benefits of getting to market sooner. It’s free money.

The more frequently you ship, the more you get, as Scenario C shows. It’s the same as Scenario B, except that the team has sliced their SIVs more finely. Instead of shipping a $4 SIV in two months, they ship a $2 SIVs in one month. After seven months, they’ve earned $42.

Of course, your SIVs don’t have equal value. Scenario D shows what happens when you ship your most valuable SIVs first. This scenario is the same as Scenario C, in that each SIV still takes one month to complete, but they’re worth different amounts. The total value is unchanged—$12 per month—but Scenario D earns $50 in the time it takes Scenario A to earn $12.

This is the essence of Focusing fluency. Focus on small, shippable increments of value. Focus on shipping one at a time. Focus on the one that’s most valuable. As each one ships, take advantage of any new information you’ve obtained, adapt your plans, and Focus on the highest value that remains.

The scenarios in the “Effects of Focusing On Value” figure are necessarily simplified. Software By Numbers [Denne and Cleland-Huang 2004] (pp. 19-24) has a more sophisticated example based on a real product, shown in the “Realistic Example of Focusing On Value” table. In their example, the authors convert a five-year project with two end-of-project releases (Scenario A) into five yearly releases ordered by value (Scenario B). The team’s productivity was the same in each scenario.

Table 1. Realistic example of focusing on value

Scenario AScenario B
Total Cost$4.312 million$4.712 million
Revenue$5.600 million$7.800 million
Investment$2.760 million$1.640 million
Payback$1.288 million$3.088 million
Net Present Value @ 10%$0.194 million$1.594 million
Internal Rate of Return12.8%36.3%

Scenario A is a marginal investment with a 12.8% rate of return. It requires an investment of $2.8 million and yields profits of $1.3 million. Considering the risk of software development, the investors can put that money to better use elsewhere. The project should not be funded.

Scenario B—the same product released more often—is an excellent investment with a 36.3% rate of return. Although Scenario B costs more, because it conducts more releases, those releases allow the product to be self-funding. As a result, it requires a smaller $1.6 million investment and yields profits of $3.1 million. This project is well worth funding.

Look at these examples again. Each shows impressive increases in value. Yet nothing changes except the way the teams release their software.

Sieve Your SIVs

As the previous scenarios show, the more finely you can slice your SIVs, the more value you can extract from your team’s work. Beyond that, though, each SIV represents an opportunity to change direction without incurring waste. If you change direction in the middle of working on a SIV, you’re left with partially-completed work you either have to set aside or throw away. (See “Key Idea: Minimize Work in Progress”.) Once a SIV is done, you can change direction without wasting any work.

The smaller your SIVs, the more frequently you can adapt your plans, and the more agile you can be. In a perfect world, your SIVs should be broken down to the smallest fundamental pieces that can still be shipped, and still have value.

In reality, it’s hard to make your SIVs that small, especially at first. As you go into more detail, you’re likely to see ways to split your SIVs further. It’s okay to start with your best guess and split them later. Agile’s iterative. You’ll have plenty of opportunities to improve your SIVs.

Release Early, Release Often

SIVs are actually shippable. Each SIV can be released on its own. Some people in the Agile community talk about “potentially shippable increments,” which have the technical capability to be shipped, but don’t necessarily have enough value to actually ship. That’s different than a SIV. If a SIV doesn’t have enough value to ship on its own, then it doesn’t qualify to be a SIV.

You can release (ship) each SIV as soon as it’s finished, or you can wait and bundle multiple SIVs together into a single release. Although bundling SIVs into a release delays their value, sometimes it can be more effective to market multiple SIVs together rather than dribbling them out one at a time. Similarly, if there are user interface changes or other release-related costs, it can be easier to absorb those costs all at once rather than continuously making small changes.

Allies
Continuous Deployment
Feature Toggles

Delivering teams that use continuous deployment release their code multiple times per day. However, they also techniques such as feature toggles to hide features from users. For these teams, “releasing” a SIV means changing a configuration setting to make the SIV available.

Sometimes teams bundle SIVs together because technical constraints make it too costly to release more frequently. It’s okay to do this, but if you do, be honest with yourself about the reasons. In addition to high release costs, you’re also incurring the costs of increased WIP. (See “Key Idea: Minimize Work in Progress”.) You can eliminate both costs by investing in Delivering fluency.

Some teams use release trains to schedule their releases. A release train is a pre-scheduled series of releases—for example, the first day of every month, or the beginning of every quarter. Any SIVs that are done “get on the train” and are shipped in that release. The rest wait for the next train. It doesn’t matter how close they are to being done; the train always leaves on time.

The Scaled Agile Framework (SAFe) defines an “Agile Release Train” as a container for teams, which is an unfortunate redefinition of the term.3 Here, I’m using the simpler, original meaning. A release train is just a predefined release schedule. Don’t overthink it.

3The term dates to well before SAFe. The earliest reference I’m aware of is Sun’s Software Development Framework from 1993, which defined the term in the same way I do here. That document isn’t publically available, but a 1998 conference paper by Johanna Rothman [Rothman 1998] references the term and uses a similar definition. Many thanks to Howard Fear and Dave Hounslow for digging up these references.

Release trains have a lot of benefits. They give marketers an event to celebrate. They give users a date to anticipate. They give stakeholders certainty about when they can expect progress, and they take pressure off teams, who—so long as their SIVs are smaller than the release cycle—can commit to release dates without worrying about exactly what they’ll ship on each date.

On the other hand, a decision to delay a SIV, either by bundling it into a one-off release or using a release train, is a decision to delay value. Organizations also tend to use release trains to paper over technical and organizational deficiencies that prevent more frequent releases. When you think about your release strategy, trade off the benefits of bundling SIVs into releases with the cost of delaying their value, and be honest with yourself about your reasons for delaying.

Your First Release

The first release can be tricky. It needs enough SIVs to be interesting, but not so many that you delay your release and the associated value.

One way to think about your first release is to think in terms of a “Minimum Viable Product” (MVP). Contrary to the common understanding of this term, an MVP isn’t the smallest product you can successfully ship. Instead, it’s a way of validating your product ideas. Eric Ries defined the term in his influential book, The Lean Startup:

A minimum viable product (MVP) helps entrepreneurs start the process of learning as quickly as possible. It is not necessarily the smallest product imaginable, though; it is simply the fastest way to get through the Build-Measure-Learn feedback loop with the minimum amount of effort.

Contrary to traditional product development, which usually involves a long, thoughtful incubation period and strives for product perfection, the goal of the MVP is to begin the process of learning, not end it. Unlike a prototype or concept test, an MVP is designed not just to answer product design or technical questions. Its goal is to test fundamental business hypotheses. [Ries 2011] (pp. 95-96)

Eric Ries

Allies
Discovery
Experimentation

An MVP isn’t necessarily a release, or even a product, in the traditional sense of the terms. It’s an experiment, and you can have more than one. As such, true MVPs are most often used by Optimizing teams.

Whether your first release is an MVP in Eric Ries’ sense of the word, or just the smallest release that customers and users will love, is up to you.

Allies
Visual Planning
The Planning Game

Adapt Your Plans

You’ll form your initial ideas about SIVs and releases before a single line of code is written. This is when you know the least about what will make the software valuable. You might know a lot, but you’ll always know more after you talk with stakeholders, show them demos, and conduct actual releases. Over time, you’ll discover that some of your initial ideas about value were incorrect. No plan is perfect, but if you change your plan to reflect what you’ve learned—if you adapt—you create more value.

To increase the value of your software, create opportunities to learn. Think of your plan as a plan for learning as much as it is a plan for building. Focus on what you don’t know. What are you uncertain about? What might be a good idea? Which good ideas can you prove in practice? Don’t just speculate—create SIVs for experiments. Include a way of testing each uncertainty.

For example, if you were creating a collaborative online word processor, you might not be sure how extensive your support for importing Microsoft Word documents should be. Some sort of support is necessary, but how much? Supporting all possible Word documents would take a long time to implement. It would take time away from adding other, possibly more valuable features. Too little support could damage your credibility and cause you to lose customers.

To test this uncertainty, you could add a rudimentary import feature to your software (clearly marked “experimental”), release it, and have it create a report on the capabilities needed to support the types of documents that real users try to import. The information you gather will help you adapt your plan and increase your team’s value.

Users of web-based software are used to “beta” web applications, so releasing an experimental, incomplete feature is possible in that context. A product with less forgiving users may require the use of a pre-release program, focus groups, or some other feedback mechanism.

Divide SIVs into Stories

Ally
The Planning Game

As you’ll see when we look at the planning game, your team should make tangible forward progress multiple times per week. It reduces risk, gives you the ability to share progress with stakeholders, and allows you to collect feedback. Ideally, that progress comes in the form of completed SIVs. The reality, though, is that most SIVs are too large, at least at first.

Ally
Stories

So instead, Agile has the idea of the “story.” A story is an increment of value, like a SIV, but it isn’t necessarily shippable. It makes progress towards something valuable, and stakeholders recognize that value, but it doesn’t always have enough value to be released on its own.

In a perfect world, every story is a SIV, and vice-versa. But there are two different goals here: First, you need to be able to show progress in small pieces. Second, you need each piece you release to be truly valuable and shippable. Sometimes you can meet both goals at once. But when you can’t, use small stories to incrementally show progress, and larger SIVs to plan releases.

Some people talk about “epics” as a way of grouping stories. (An epic is a big story. Get it?) I use SIVs instead, because “big” isn’t the important part. SIVs stay focused on what matters: they’re shippable; they’re incremental; they’re valuable.

How to Create Your Plan

It takes a lot of time and effort to come up with SIVs, break them into stories, size them, and prioritize them. If you’re adapting your plans as you go, some of that effort will be wasted when your plans change.

Ally
Incremental Requirements

To reduce waste, use rolling-wave planning to plan at the last responsible moment (see “Key Idea: The Last Responsible Moment”). In rolling-wave planning, only work that you’re about to do is planned in detail. Work that’s going to be done later is planned approximately. As work gets closer, the plans are refined and broken down into more detail.

A chart showing a funnel on its side. The wide end of the funnel is on the left, labelled “approximate, broad view,” and the narrow end is on the right, labelled “detailed, narrow view.” The funnel is divided into several regions. Each region is labelled, and the regions are separated by a dividing line with a time and the caption “time until work is done.” From left to right, the regions and dividing lines are: “purpose” (region), “six months” (divider), “high-level SIVs” (region), “three months” (divider), “small SIVs” (region), “one month“ (divider), “stories” (region), “one week“ (divider), “tasks and detailed requirements” (region), and “done“ (final divider).

Figure 2. Planning horizons

The distance each level of detail looks into the future is called its planning horizon. An Agile plan has multiple planning horizons, as illustrated in the “Planning Horizons” figure:

Allies
Purpose
Visual Planning
The Planning Game
Task Planning
Incremental Requirements
  1. Start with the purpose for the team.

  2. Use visual planning to create a map of high-level SIVs.

  3. Break the first few high-level SIVs down into the smallest possible SIVs.

  4. Use the planning game to break the first few small SIVs into stories.

  5. Use task planning to break the first few stories into development tasks.

  6. Just prior to starting development on a story, use incremental requirements to determine details.

As your team completes work, you’ll “pull” from the next level up. When you finish your tasks, you’ll use task planning to break the next stories into tasks. When you need more stories, you’ll use the planning game to break the next SIVs into stories. When you need more SIVs, you’ll use visual planning to refine your SIVs and create new ones. As you get close to finishing your SIVs, you’ll go back to your sponsor to create a new purpose.

Different skills are needed for each level of detail, as the “Planning Skills” figure shows.

This chart shows the same image as the “planning horizons” figure, except the funnel and regions are in the background. The names of several skills are overlaid on the funnel, with arrows showing which regions the skills apply to. From left to right, they are: “sponsor and key stakeholders,” which overlays the “purpose” and “high-level SIVs” regions; “product management,“ which overlays “purpose” (partially); “high-level SIVs,” “small SIVs,” and “stories” (partially); “domain expertise and UX design,” which overlays “high-level SIVs” (partially), “small SIVs,” “stories,” and “tasks and detailed requirements”; “business-focused testing,” which overlays “small SIVs (partially), “stories,” and “tasks and detailed requirements”; and finally “programming, operations, and technology-focused testing,” which overlays “small SIVs” (slightly), “stories,” and “tasks and detailed requirements.”

Figure 3. Planning skills

Balancing Adaptability and Predictability

Each level of detail in your plan has an associated planning horizon. For example, the “Planning Horizons” figure shows that tasks are planned for the next week; stories are planned for the next month; small SIVs are planned for the next three months; and high-level SIVs are planned for the next six months.

Your planning horizons are a tradeoff between adaptability and predictability. For example, if you’re likely to change your plans, you might plan stories just two weeks in advance, and small SIVs just a month in advance. On the other hand, if your stakeholders need a lot of certainty, you could plan stories three months in advance, and small SIVs six months in advance.

Allies
Roadmaps
Forecasting

The longer your planning horizons, the more work you’ll throw away when things change, and the more people will resist making changes. (People get attached to plans.) On the other hand, longer planning horizons are often needed for good roadmaps, and your story planning horizon determines how far into the future your release forecasts can see.

Ultimately, the choice of planning horizons is a tradeoff between less waste and greater agility (shorter planning horizons) and more certainty and predictability (longer planning horizons). There’s no wrong answer; just a choice between tradeoffs. If you aren’t sure what to choose, start with the horizons shown in the “Planning Horizons” figure.

Adaptive Planning and Organizational Culture

Does the idea of spending two months travelling in a foreign country without a detailed plan sound scary? In practice, it was easy and relaxing, but when I tell the story of our adaptively-planned trip to Europe (see the “Adaptive Planning in Action” sidebar), some people get nervous.

Organizations often have a similar reaction to adaptive planning. An adaptive plan works to achieve the team’s purpose. However, just as the trip my wife and I took achieved its purpose—“have fun visiting a lot of European cities”—without choosing specific cities in advance, an adaptive team will achieve their purpose even though they won’t be able to say exactly what they’ll deliver.

No aspect of Agile challenges organizational culture more than the transition to adaptive planning. It requires changes not only to the development team, but to reporting, evaluation, and governance. The choice of adaptive planning extends to surprisingly diverse parts of your stakeholder community, and people often have a shocked or emotional reaction to the idea.

As a result, you may not be able to influence a change to adaptive planning. Unless you have the support of senior leadership, any change that does occur will probably be slow and gradual. Even with leadership support, this change can take time.

Ally
Forecasting

Work within your organization’s culture to introduce adaptive planning gradually. Use rolling-wave planning, but set your planning horizons to match the organization’s expectations. You may need to make forecasts as well, which typically requires Delivering fluency. As your stakeholders gain trust in your ability to deliver, shorten your planning horizons and migrate towards a more adaptive plan.

Questions

We need to commit to specific release dates. What should we do?

There’s no problem with defining a release date in advance. You won’t know exactly what will be in that release, but you can adjust your plans as you go to have at least one SIV ready to release on time.

Ally
Forecasting

If you need to plan for specific SIVs on a specific date, or predict when a SIV will be done, you can use the forecasting practice to do so. You’ll typically need Delivering fluency for your forecasts to be useful.

If we don’t plan our releases in detail, what should we tell our stakeholders about our plans?

Allies
Roadmaps
Purpose

Although you may not plan out all the details of your releases in advance, you’ll still be able to share a roadmap with stakeholders. At a minimum, you’ll be able to tell them which SIV you’re working on and what’s currently planned to come next. You may have a pre-planned release date, and, if you do, you can share that date with confidence, along with any SIVs that are finished and ready to be part of that release.

Planning at the last responsible moment means we can’t show exactly what we’ll deliver. Doesn’t that require too much trust from stakeholders?

Allies
Trust
Forecasting

Any development effort requires that stakeholders trust the team to do its job. In many organizations, though, that trust has been lost. If stakeholders require a detailed plan in order to trust you, use longer planning horizons that allow you to provide the plan they desire. Developing Delivering fluency can help, too, because that will allow you to use forecasts to make and meet commitments more reliably.

If we use short planning horizons, how can we be sure we’ll fulfill our team’s purpose?

If you’re not sure you can meet your purpose, focus your plan on discovering whether you can. You may need to create SIVs that are experiments, extend your planning horizons, or create a small, limited-availability release to test crucial concepts. The details depend on your situation, so if you’re not sure what to do, ask a mentor for guidance.

No matter your decision, clearly convey your concern to your business stakeholders, and let them know how you intend to address the uncertainty.

Prerequisites

Adaptive planning requires manager and stakeholder buy-in (see the “Invest in Change” chapter). The one thing that any team can do is to plan in terms of SIVs and stories. From there, it depends on how much buy-in you have.

Working on one SIV at a time is a smart, easy way to increase your team’s value. Despite its usefulness, working on one thing at a time is anathema to some stakeholders. Proceed with caution.

Releasing frequently requires that your customers and users be able to accept more frequent releases. This is a no-brainer for most web-based software, because users don’t have to do anything to get updates. Other types of software may require painful software rollouts, and some even require expensive certification testing. That can make frequent releases difficult.

Creating experiments, options, and MVPs requires that your organization accept uncertainty and trust that your team has sufficient market expertise to make good decisions. This often requires your team to have Optimizing fluency.

Allies
Purpose
Whole Team

Rolling-wave planning requires a clear purpose and regular updates to the plan. Use it when you can reliably revisit the plan at least once per week. Be sure your team includes people with product management and other customer skills who are responsible for maintaining the plan.

Adapting your plans requires that your organization think of success in terms of value rather than “delivered on time, on budget, and as specified.” This can be a tough idea for some organizations to swallow. You may be able to assuage fears about adapting plans by committing to a specific release date but leaving the details of the release unspecified.

Finally, be cautious of plans without concrete goals in the next three months, such as a release or mission test (discussed in the “Purpose” practice). Although those goals shouldn’t be treated as commitments, it’s easy to wander off course without the checkpoint and urgency of a near-term goal.

Indicators

When you create, maintain, and communicate a good plan:

  • The team and its stakeholders all know where the team is heading.

  • The plan shows how the team will achieve its purpose, or learn how to achieve it.

  • Team members are confident the plan is achievable.

  • You ship value regularly and consistently.

When you adapt your plan well:

  • You consistently seek out opportunities to learn new things about your plan, your product, and your stakeholders.

  • As you learn, you modify your plan to take advantage of new insights.

  • Stakeholders and the team agree that each release is better than originally planned.

Alternatives and Experiments

Adaptive planning increases value by being flexible in planning and strategic in releases. Look for opportunities to reduce time spent on plans that are discarded, quicken the feedback loop, improve your plans more often, and shorten time to value.

Ally
Evolutionary Design

If you don’t have a Delivering team, you might run into questions about how to plan for technical infrastructure. [Denne and Cleland-Huang 2004] provides a sophisticated Incremental Funding Methodology that addresses that question. Teams with Delivering fluency sidestep this need, because they use evolutionary design to build their technical infrastructure incrementally.

Teams with an established product don’t always need a sophisticated plan. Rather than thinking in terms of SIVs or releases, these teams work from a small list of stories and constantly release minor changes. You can think of this is an adaptive plan with very short planning horizons.

Finally, adaptive planning is often seen as an alternative to predictive planning, but as the “Balancing Adaptability and Predictability” section shows, it’s really more of a spectrum of different planning horizons. If you’re in an environment that needs predictive planning, see how many adaptive ideas you can use along with your longer planning horizons.

Further Reading

Software By Numbers [Denne and Cleland-Huang 2004] provides a compelling and detailed case for conducting frequent releases.

Agile Software Development Ecosystems [Highsmith 2002] has an excellent discussion of adaptive planning in Chapter 15.

Lean Software Development [Poppendieck and Poppendieck 2003] discusses postponing decisions and keeping your options open in Chapter 3.

XXX Consider Reinertson, The Principles of Product Development Flow: Second Generation Lean Product Development

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Chapter: Planning (Introduction)

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Planning

Agile is adaptive, not predictive. It’s one of the things that sets Agile apart—the very source of its name!—and one of the biggest culture shocks for organizations new to Agile. And nowhere is it more apparent than in the way Agile teams plan their work.

This chapter has the practices you need to effectively make and adapt your plans. Because adaptive planning can take time for organizations to accept, it also discusses how to make predictive plans with an Agile team.

  • The “Adaptive Planning” practice: Balance adaptability with predictability to focus your plans on value.

  • The “Visual Planning” practice: Create plans that communicate context and options.

  • The “The Planning Game” practice: Create a detailed plan for guiding your next steps.

  • The “Stories” practice: Plan your work in small, customer-centric pieces.

  • The “Incremental Requirements” practice: Define requirements in parallel with other work.

  • The “Real Customer Involvement” practice: Include customers’ perspectives to create better plans.

Adaptability is supported by several key ideas. They’re discussed in sidebars throughout this chapter:

  • “Key Idea: Minimize Work in Progress”: Decrease waste and improve agility.

  • “Key Idea: The Last Responsible Moment”: Make better decisions by deciding later.

  • “Key Idea: Maximize Work Not Done”: Save time by skipping unimportant work.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

Coding Over Cocktails Podcast

I appeared on the Coding Over Cocktails podcast earlier this month. It’s a nice, free-ranging conversation covering topics from large-scale Agile development, to complexity, to organizational investment.

Check out the excerpt in the video below, or listen to the whole podcast at one of these links:

AoAD2 Chapter 6: Invest in Change

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Invest in Change

You’ve decided that Agile will make your teams more successful. You know which zones have the best cost/benefit tradeoffs. You’ve figured out which investments your company needs to make. Now, how do you make it happen?

Understanding Change

A chart with two axes. The horizontal axis is labelled “Time,” with an arrow pointing to the right. The vertical axis is labelled “Performance,” with an arrow pointing up. Within the chart, a line travels from left to right. The line starts in the middle of the performance axis with a section labelled “late status quo.” It is mostly horizontal, with small vertical pertubations and a slight upward trend. Next, there’s a star labelled “foreign element.” The line continues as before through a short section labelled “resistance,” before plunging down (indicating lowered performance) with dramatic up-and-down spikes (indicating highly-varying performance]. This section is labelled “chaos.” None of the spikes exceed the midline performance of the late status quo. About halfway along the horizontal axis, one of the spikes is labelled with another star and the title “transforming idea.” The spikes gradually climb, indicating increased performance, and the size of the swings decrease. This section is labelled “integration.” Finally, the spikes flatten out, at a higher level of performance than the beginning, with small vertical pertubations and a slight upward trend. This final section is labelled “new status quo.”

Figure 1. The Satir Change Model

Change is disruptive, and moving to Agile is no exception.

Change is disruptive, and moving to Agile is no exception. Exactly how disruptive it is depends on how many teams are affected and how well you manage the change. If you have one team that’s eager to try Agile with your organization’s full support, it doesn’t have to be a big deal. If you’re trying to change 50 teams in an organization that’s unfamiliar with Agile ideas... well, now it’s a very big deal.

One way to understand how people respond to change is Virginia Satir’s Change Model, shown in the “Satir Change Model” figure.1 As the figure shows, there are five stages to a change. Here’s how they apply to Agile:

1Steven Smith has a good article on the Satir model at https://stevenmsmith.com/ar-satir-change-model/ that includes tips for helping team members through each stage.

  1. Late Status Quo. This is the pre-Agile way of working. It’s comfortable and familiar. Everyone knows what’s expected of them and how to do their job. Some people aren’t completely happy, though, and they think Agile will help. They push for change.

  2. Resistance. The people who want change start getting traction, and some sort of Agile change becomes likely. This is called a “foreign element,” and people start responding to the possibility of change. Many will oppose it. They’ll say changing to Agile is unnecessary, unlikely to succeed, or a waste of time. Some will be angry. The more people that are affected, the more resistance you’ll see.

  3. Chaos. The Agile change is approved and teams start using Agile practices. Old ways of working and familiar expectations no longer apply. People feel lost and confused, and moods are volatile. Some days are good; some are bad. People occasionally revert to childish behavior. Performance and morale decline.

  4. Integration. With practice, people start to become familiar with their new ways of working. They discover an aspect of Agile—called a “transforming idea”—that is particularly compelling to them. (This will be different for each person.) They embrace the possibilities Agile brings and start putting real effort into making Agile work. Feelings of chaos decrease, morale improves, and performance climbs.

  5. New Status Quo. People have made their way through the change and come out the other side. Their new Agile ways of working are comfortable and familiar, and they’re confident enough to continue to make small changes. Performance has stabilized at a higher level than before the change, and continues to increase gradually as people experiment with further small changes.

Trying to rush change just makes things worse.

This reaction to change is unavoidable. Trying to rush it just makes things worse. That’s why organizations need to set aside time for learning Agile (see the “Make Time for Learning” section). Note the parallel between the Satir model shown in the “Satir Change Model” figure and the J-curve shown in the “Agile Performance Over Time” figure.

Everybody goes through these stages at their own pace. The length of the change, and the depth of the chaos, depends on how much their day-to-day life is affected. Someone who’s only peripherally involved will respond less than a person who’s part of a newly-Agile team. Individual personalities matter, too; some people love trying new things, while others want stability and predictability.

You can decrease (but not eliminate!) the chaos by using a technique I learned from Diana Larsen: Support, Information, and Structure (SIS).2

2Thanks to Diana Larsen for assisting with this list.

  • Support. Help people understand how to do their job in the changed environment. Provide training, coaching, and other ways for people to get help without feeling judged. Make the investments described in the “Invest in Agility” chapter. Make sure everyone has someone they can talk to, either at work or in their personal life, when they’re feeling overwhelmed.

  • Information. Be transparent about what’s happening, what’s known, and what’s yet to be determined. Address people’s career concerns. If you can do so honestly, make an explicit promise that no one’s getting fired as a result of the change. Communicate more than you think should be needed.3

  • Structure. People need ground to stand on, so provide a roadmap for the change. If you’re using this book as the basis for your change, provide copies, and tell people which parts you’re using. When things are uncertain, describe what you need to do to make them certain, and when you expect that to happen. If there’s an interim step, such as temporary teams, be clear that it’s temporary, and describe what’s going to happen next.

3Diana says, “Communciate until you want to throw up. And then more.”

Large-Scale Change

Changes that affect a lot of people are much more disruptive than changes that only affect a few teams. The disruption is multiplicative. Rumors start flying, people start worrying about their jobs, and the upcoming changes become part of every conversation.

Large changes—those that directly impact more than 30-70 people—require professional change management. Depending on the size of your organization, your HR department may have change management staff who can help. If not, you can hire consultants. Some Agile consultants have change management experience and can guide both aspects of your Agile adoption. For very large changes that directly impact more than 200-500 people, you’ll need dedicated change management experts to work alongside your Agile guides.

Organizational leaders often underestimate the importance of change management. This is a grave mistake! To put it in terms of the Satir model, everybody goes through a period of resistance and chaos when they learn about a change. But everybody goes through it at their own pace, and leaders typically go through it first. Not because they’re better or more resilient, but because they know about it first.

By the time the rest of the organization learns about the change, organizational leaders have already experienced the “transforming idea” that resolves their feelings of resistance and chaos. To the leaders, the change now seems obvious and necessary. Why would anybody disagree?

Then they introduce the change and experience massive amounts of pushback and disruption from people going through their own resistance and chaos stages. It can kill the change entirely.

Don’t skimp on proper change management.

Proper change management can’t prevent all disruption, but it can reduce it. Don’t skimp. If you’re not prepared to get expert help, limit your Agile changes to a few teams at a time.

Making Changes

Kaizen (rhymes with “I win”) is a common term in the Agile community. It’s a Japanese word meaning “improvement.” In the Agile community, it specifically means continuous, gradual improvement.4

4Kaizen was imported to Agile from Lean Manufacturing, which itself is based on the revolutionary Toyota Production System—hence the Japanese terminology.

Continuous improvement is an integral part of Agile, so shouldn’t you kaizen your way to Agile in the first place? Counterintuively... probably not. Kaizen is for improving your existing ways of working. If you have a document-oriented culture, kaizen will help you streamline your documents. If you have a blame-oriented culture, it will help you place blame more accurately. But it won’t help you make the leap from either to Agile.

To jump from one way of working to another, you need a different Japanese word: kaikaku (rhymes with “I rock you“). Kaikaku means “transformative change.” Instead of incrementally improving your existing way of working, as kaizen does, kaikaku fundamentally changes your underlying approach.

The great Agile teams I know all started with kaikaku. They figured out what they wanted from Agile, how they were going to invest in getting those results, and they went all in.

I see many more mediocre Agile teams than great Agile teams. One thing the mediocre teams have in common is that their companies didn’t go all in. They tried to kaizen their way to Agile. It seems to work at first, but invariably stalls out. People burn out on the mismatch between Agile ideas and company values. They get tired of absorbing new ideas. Change fatigue sets in and progress stutters to a halt after several years of effort. Ironically, the disruption from approaching Agile this way lasts much longer than the disruption from kaikaku.

If your company is new to Agile ideas—regardless of whether they already use the name—use kaikaku. Choose your zones, make the investments, and have each team start using all the corresponding practices at once. It may seem scary, but it’s actually faster and safer than adopting practices gradually.

If you’d like to take an incremental approach, kaikaku is still your best bet. Rather than using kaizen to gradually introduce Agile to a lot of teams, use kaikaku to fully introduce Agile to one team. For an even smaller increment, start with the Focusing zone alone. Then add the Delivering zone. Then add another team, perhaps with Focusing and Delivering together. As you gain experience, you can increase the size of your increments.

Teams that are already Agile can kaizen their way to better results within their current fluency zones. See the “Improvement” chapter for details. To move to new zones—for example, if a team is fluent in the Focusing zone, and wants to add Delivering or Optimizing fluency—kaikaku is still the best approach. New zones require new investments and major changes, and that’s best done all at once.

Successful kaikaku requires discipline and care. And it starts with...

Get Management Buy-In

Agile requires management support. Without it, the mismatch between your teams’ Agile practices and your organization’s non-Agile culture will cause constant friction. If you’re a manager yourself, you still need your manager to be on board, and it’s best if your peers support you, too.

1. Start with a conversation

It begins with a conversation. This is often easiest one-on-one, and you’ll be most successful if your conversations are in-person, or at least over video. Start with an influential manager you trust and recruit them as an ally. They’ll help you understand who else you need to talk to and how best to approach them.

In your conversations, starting with that first manager, talk about the challenges your organization faces with software development. Based on the benefits described in the “Choose Your Agility” chapter, talk about how you think software development could be better in your company. Don’t monopolize the conversation, though; engage your audience. Briefly share the benefits of each zone and ask which zones they think are important. Ask them why. Spend more time listening than talking.

Above all, focus on what they can get and what inaction could cause them to lose rather than pushing Agile for the sake of Agile. In fact, given the extent of misunderstandings about what Agile is, you might be better off not even mentioning the word “Agile.”

2. Get the economic buyer’s approval

Your ultimate goal is for you to speak to the person who has the authority to make the investments your teams need. In sales, this person is called the “economic buyer.”

Economic buyers are often surrounded by gatekeepers who see their job as protecting the economic buyer. They’ll ask you to make your pitch to them, so they can present it to the buyer. They’re not trying to steal your idea; they’re trying to save the buyer time. Sometimes they’ll assure you that they’re the real buyer, even if they don’t actually have the authority needed.

Don’t be fooled. Although it’s helpful to get gatekeepers on your side, and often necessary, it’s not enough. Gatekeepers can’t approve the investments you need. You need to talk to the real economic buyer.

Prior to talking to the economic buyer, focus your conversations on the benefits of Agile: what’s at stake. The investments are likely to be a distraction, or even a source of concern, because the people you’re talking to won’t have the authority to make those investments.

When you finally talk to the economic buyer, your goal is to get them to agree in principle to investing in Agile. You probably won’t have a lot of time, so stay focused on the big picture. It’s also often best if you approach your meeting as a conversation, rather than a presentation with slides, but your allies will know the best approach for your specific situation.

In your conversation with the economic buyer, talk to them about what they want from their organization and how Agile will help. This works even better if a manager they trust speaks informally on your behalf.

Once the buyer is on board with Agile’s benefits, talk about the investments. Don’t overwhelm them with detail; just summarize a few key investments needed for each zone (the “Summary of Investments” sidebar will help you prepare), along with how those zones map to what the economic buyer wants, and ask them which investment/benefit tradeoffs seem most appropriate. Giving them a choice of several options, rather asking for a yes/no decision, reduces the chance that they’ll reject you outright.

Assuming the economic buyer agrees in principle to investing in Agile, or that it’s at least worth considering further, ask for permission to create a concrete proposal. Ask what they need to see in the proposal, in general, for them to approve it. Ask for them to recommend a sponsor for you to work with, provide a date when you’ll have the proposal to them—within a day or two is best, so it’s a good idea to have a rough draft done already—and ask when you should expect to hear back.

Finally, ask for permission to follow up with them. They probably won’t get back to you when they said they would, so it’s good to be able to say, “I’m following up as you requested.”

3. Make a formal proposal

If you got this far, congratulations! You’ve passed your most important hurdle. Now you need to follow through with a proposal.

The formality of your proposal will depend on your organization. Your sponsor and other allies will help you understand how to format your proposal, help you refine it, and promote it to the economic buyer. Be prompt, polite, and persistent.

In your proposal, describe the benefits your that organization can expect to see and the investments it needs to make. Be concrete. The “Why Agile” chapter describes Agile’s benefits in general, and the beginnings of parts 2-4 go into more detail. Translate those benefits to your actual situation, the investments the economic buyer is willing to make, and what that realistically means for your organization.

For the investments part of your proposal, read the “Invest in Agility” chapter and translate each step into concrete requests. You may have to compromise on some investments. That chapter explains how. But avoid compromising too much. The investments are ultimately what make Agile’s benefits possible.

If this sounds like too much work...

This careful buy-in process is for when support is uncertain: when you’re working with multiple teams, asking for big investments, or working in a bureaucratic organization that’s uncomfortable about Agile ideas (even if they use the name a lot).

But sometimes, that’s not the case. Sometimes you’re just helping one small team become more Agile. If you and your manager already have the power to make the investments you need, do it!

If management thinks they’re already Agile...

Some organizations—these days, a lot of organizations—think they’re already Agile. One company told me, “We’re post-Agile!” Or you might hear, “we’re little-a agile, not big-A Agile!” But when you compare the philosophy in the “What Is Agile” chapter to how the organization acts, and the benefits in the “Why Agile” chapter to what they get, you see that they’re nowhere close.

Stay focused on the situation at hand: challenges, benefits, and investments.

There’s nothing to be gained from arguing about terminology. If they want to say they’re agile, or post-agile, or extra-super-duper agile, let them. Instead, stay focused on the situation at hand: the challenges your teams are facing. The benefits the organization could get. The investments needed to get those benefits.

If management isn’t supportive...

If you can’t get traction with managers at first, don’t give up. Put yourself in their shoes. What does organizational success mean to them? What about personal success? How can Agile help them get what they want? If the answer is “it can’t,” then maybe Agile isn’t a good fit. Choose another approach to software development, one that better fits your organization’s culture. See the “Succeeding With Waterfall” sidebar for one possibility.

If you have a trusted manager you can turn to, ask for help and advice. If not, try conducting an informational interview with a manager at a company who’s been through it before. (They may try to recruit you. Win/win!) The “Change Your Organization” sidebar has more ideas.

In the early days of Agile, when it was a grass-roots movement, many teams adopted Extreme Programming on their own, with little to no permission or support. You could try that. But I don’t recommend it. In the experience reports from teams who tried it, somebody—often the project manager—ended up having to bridge the gap between company culture and Agile philosophy. It was a thankless job and burned them out.

Some people use the Kanban method to motivate organizational change.5 Kanban wraps around existing ways of working to highlight work-in-process bottlenecks and the cost of delay. It’s easy to put in place and can motivate a more Agile approach to your work.

5Note that the Kanban method is much more than just the Kanban board some teams use for planning.

Kanban is a kaizen approach to change, so it’s slow and can only go so far, but it’s very effective and can lead to permission for kaikaku. See XXX [choose book recommendation] for more information.

If nothing you do makes a difference, take a hard look at what you need for personal success. Assume the status quo won’t change, because it probably won’t. Either that’s good enough for you—and it often is—or it’s time to move to a company that better fits your needs.

Get Team Buy-In

Agile puts people first, so it shouldn’t be a surprise that you need your prospective Agile teams to agree to trying Agile. It is possible to force people to nod agreement with gritted teeth, but—and I speak from hard-won experience, here—that road involves a lot of employee turnover. Or it fails, with a passive-aggressive whimper.

When I’m asked to help teams become Agile, I always speak to each team on their own, without managers present. You want them to be comfortable expressing themselves without fear of retribution. Include their coach, and if you’re a manager yourself, start the meeting by expressing your support for their decision—whatever it may be—then let them speak to the coach without you.

When you or the coach talk to each team, explain that their team has been chosen as a possible candidate to try Agile. I explain why their managers are interested in Agile, what benefits it will bring to the organization, and how it will affect them personally. I also explain that changing work habits can be stressful, and that they should expect a period of chaos—typically, up to three months—as everyone figures out how to make Agile work for them. I often sketch out and describe the Satir model (see the “Satir Change Model” figure).

“If you agree,” I tell them, “I’ll ask you to follow a by-the-book approach to Agile for three months. At that point, we’ll evaluate what’s working, what’s not, and make improvements.6 After six months, you’ll have the final say on whether you continue with Agile or go back to what you have now.”

6This is a little white lie. Agile involves constant improvement, so we evaluate what’s working and what’s not within a few weeks. But we do pause for a bigger evaluation at the three month mark.

Then I open the floor for questions. Teams typically a lot of questions about the process, but invariably, one of the questions is, “What happens if we say no?” My answer is always the same: “Then it doesn’t happen.” This is important! The veto must be real. If you don’t give people the option of saying, “no,” their “yes” doesn’t mean anything. By giving people a real chance to refuse now, and a concrete opportunity to change their minds later, you make it safe to try something new.

Make sure you allow plenty of time to answer everybody’s questions. The meeting usually takes about an hour, but sometimes it goes longer. After everyone’s questions have been addressed, remind everyone that there’s no consequence for voting against Agile. Mean it. Then ask the team to vote.

If team members are skeptical...

Skepticism is normal, and you should expect it. Be straight with your teams: change is disruptive, but the results are rewarding. Be clear about the practices that you think people might find frustrating or unusual at first, such as pair programming. That will help disarm skepticism and also make it easier to introduce those practices in the future.

It helps to emphasize that this is an experiment, and that the team has the final say over whether they stick with Agile or not.

If a few team members refuse...

If a few people disagree, ask them to explain why, and see if their objections can be resolved. If they can’t, ask if they’d be willing to reserve judgement and go along with the rest of the group for six months.

If they still don’t agree, consider asking if they’d be okay moving to another team, subject to management approval. If that’s not an option, or if you don’t know who disagreed (in the case of an anonymous vote), then this team isn’t a good candidate.

If the majority of the team refuses...

If the team doesn’t agree, then you’ll have to choose another team. I’ve only had this happen rarely, but it does happen. In one case, it was because the team didn’t trust their organization to give them the time they needed to learn. In hindsight, they were right, and it’s a good thing we didn’t go through with the change.

If people lie about their acceptance...

Sometimes, people will vote for Agile while secretly opposing the changes. Other than making sure people don’t feel coerced, there’s nothing you can do about it. It’s not productive to second-guess people’s votes.

The nature of change is that everyone has second thoughts once in a while.

Even if nobody lies, the nature of change is that everyone will have second thoughts once in a while. You’ll need to work through those objections as they arise. When they do, it helps to be able to remind people that they agreed to stick with the experiment for six months, but there is a clear end date, and if it still isn’t working by then, they’ll be able to change their mind. Be compassionate and respectful; changing habits is takes time, and people can feel like the norms they rely on have been set adrift.

In my experience, by the time the six month date rolls around, the chaos of change will be past and people will be happy with their Agile method. That’s been true for every team I’ve worked with.

Get Stakeholder Buy-In

Your teams’ stakeholders are everyone who is affected by, or has an influence over, their work. For you and your Agile improvement efforts, your stakeholders also include anybody else who has influence over the changes you want to make.

We’ve already talked about team member and manager buy-in. You don’t need all your remaining stakeholders to buy in to your Agile intentions, but you do need the ones with a lot of political influence to do so. If they don’t, they could quietly sabotage your efforts, pulling the rug out from under you in six months to a year, even if—or especially if—Agile is succeeding.7

7Alistair Cockburn calls this “organizational antibodies.” The more successful a change initiative is, the more people worry that it will affect them, and the more they fight against it.

The stakeholders who are most likely to resist are your teams’ business partners: product management, marketing, and sales. Agile represents a major shift in how teams interact with these groups. They’re used to a predictive approach, with a focus on commitments and deadlines. Their interactions with development teams are typically focused on documents, progress reports, and sign-offs.

Agile teams focus on feedback, frequent delivery of value, and adapting their plans. They constantly ask their stakeholders for feedback, then change their plans based on what they learn. Because their plans are always changing, they don’t make detailed release commitments. Some teams do provide release forecasts, but even then, those forecasts aren’t commitments, and they change frequently.

Some stakeholders love it. Finally, they know what’s really going on, and they have the ability to influence the outcome. Others, particularly those who have been burned by missed commitments in the past, see Agile as a political maneuver: a complicated way for teams to avoid making promises. They fight Agile tooth and nail.

Talk to your teams’ key stakeholders about Agile. This is often best done one-on-one. The topic can be politically fraught, so make sure you plan your strategy with your management allies. You might not be the best person to have the conversations—a manager or person your stakeholders trust might be a better choice.

When you (or your proxy) talk to stakeholders about Agile planning, be sure to put yourself in their shoes. Demonstrate that you know what’s important to them. Perhaps they need to make marketing plans, or talk to sales prospects about competitive new features, or coordinate delivery with a third party. You want to make a connection: to dispel their preconceptions that you’re just another arrogant engineer here to tell them they can’t have what they need.

Talk about what you’re going to do better. The “Planning” chapter describes how it works. Don’t sugar-coat or over-promise. Instead, say something along the lines of, “We’ve found that, when we fix our plans in advance, we end up disappointing customers with delays or half-baked results. That’s not good for you or our reputation as a company. Instead of making a fixed plan and surprising you with delays, we’re going to give you more visibility and control, so you can see problems early, and we’ll adjust our plans as we go to get better results.” For many stakeholders, it’s not missed deadlines that are the biggest problem; it’s being blindsided by them.

Treat your stakeholders as trusted partners. You want them to be successful.

Throughout each conversation, treat your stakeholders as trusted partners. You want them to be successful. You’ve got to balance multiple interests, and you’re providing visibility and control, not predictability, but you’re there to help. You’re going to do everything you can to make their job easier and more successful.

If concrete commitments are required...

The Agile way is adaptive planning: changing your plans as you go to achieve the best possible results. That’s incompatible with making precise commitments in advance. You can commit to a specific date and steer your plans to meet that date, but you can’t predict exactly which features will be done on that date.

If that’s not good enough, you can use the approach described in the “If Waterfall Governance Is Required” section to make fixed-date, fixed-scope plans. They’re not guaranteed to be correct, but they’ll be at least as good as what you have today. If that’s still not enough, Agile isn’t a good fit.

If stakeholders don’t buy in...

Some software teams have a contentious relationship with their stakeholders, particularly those in product management and sales. It can get pretty acrimonious. In some cases, bad blood and a lack of trust might lead stakeholders to flat out refuse to support an Agile process. They might also object to the initial slow-down involved with learning Agile (see the “Make Time for Learning” section).

If only a few stakeholders object, you can choose teams that they’re not involved with. If a lot of stakeholders object, or if their high-level leadership objects, you might be able to convince them to try out a pilot with a single team. If you can, choose a team whose stakeholders are both influential and eager to try out new ideas. It might take a while—even a year or two—but they’ll come to like the visibility and control Agile gives them, and they’ll convince their colleagues to give Agile a chance.

Sometimes software organizations try to force Agile on their stakeholders. They can even succeed, if they have enough political power, but it leads to long-term blowback. If you face widespread, active opposition, to the point that even a pilot team isn’t acceptable, Agile isn’t a good fit for your organization. See the “Succeeding With Waterfall” sidebar for another option.

Further Reading

XXX Reading recommendations TBD.

Consider:

  • Agile Conversations

  • Agile Adoption Patterns (Elssamadisy)

Thomas Owens recommends:

  • Getting to Yes

  • Getting Past No

  • Crucial Conversations

Jason Yip recommends:

  • 7 Rules for Positive, Productive Change

  • Scaling Up Excellence

  • Influencer

  • https://medium.com/@jchyip/my-preferred-models-and-strategies-for-facilitating-large-scale-change-a0e498169e67

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Chapter 5: Invest in Agility

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Invest in Agility

As we saw in the previous chapter, in order for your teams to get the benefits of Agile, your organization has to buy in to the underlying Agile philosophy. Not just spending money—that’s comparatively easy—but making real, meaningful changes to organizational structures, systems, and behaviors.

If that sounds like a lot of work... well, that’s because it is. Are these investments really so important?

Yes. They really are.

Most of what holds teams back isn’t the processes they use; it’s the constraints they’re under.

Investing in Agile is important because you’re investing in changing your constraints. Most of what holds teams back isn’t the processes they use; it’s the constraints they’re under. Make the investments and ignore the practices, and your teams are still likely to improve. Perform the practices and ignore the investments? They’ll struggle.

As Martin Fowler said:1

1Excerpted from Fowler’s “Enterprise Rails” article at http://martinfowler.com/bliki/EnterpriseRails.html.

I see a startling parallel between DHH [David Heinemeier Hansson, creator of Ruby on Rails] and Kent Beck [creator of Extreme Programming]. For either of them, if you present them with a constrained world, they’ll look at constraints we take for granted, consider them to be unessential, and create a world without them... they just stick some intellectual dynamite under them and move on. That’s why they can create things like Extreme Programming and Rails, which really give the industry a jolt.

Martin Fowler

Make the investments. They’re the secret to Agile success.

The following sections describe the investments your teams need from your organization. You may not be able to get all of them, so I’ve provided alternatives. But the alternatives come at the cost of reduced effectiveness, so work hard to get as many as you can. I’ve only included the ones that are important.

Make Time for Learning

Changes are disruptive, and new ideas take time to learn. Learning Agile will slow your teams down at first.

How much will they slow down? There’s no objective measure of software productivity [Fowler 2003], but from experience, I’d estimate a 10-20% performance dip at first. As they become proficient with Agile skills, their performance will increase. It will continue to increase until they achieve fluency, and then the increase will gradually level off, as the “Agile Performance Over Time” figure illustrates. This is called “the J curve,” and it’s common to all significant changes. We’ll look at it closer in the “Invest in Change” chapter.

A graph showing how an Agile team’s performance changes over time. It shows an Agile change starting out at the same performance level as the status quo, which is marked “Agile introduction; preparing for change.” Then the performance of the Agile change drops rapidly, a period marked “Introducing practices; some resistance to change.” Performance stays low for an unspecified period of time, which is marked “Initial learning; chaos of change.” Then it rises in the shape of an S-curve, which is marked “Gaining proficincy; becoming comfortable with change.” The S-curve crosses the status quo, showing the Agile change’s performance surpassing the status quo, then gradually levels off in a period marked “Reaching fluency; change established.” Finally, it continues improving at a gradual rate, which is marked “Continuous minor improvements.”

Figure 1. Agile performance over time

The time investment will typically pay for itself in the first year. The length of the initial dip depends on the fluency zones each teams is learning:2

2These time frames are ballpark figures based on my experience. Your experience may be different.

  • Focusing: one to four months

  • Delivering: two to six months

  • Optimizing: one to three months

These periods overlap, so a team learning Focusing and Delivering skills together will have a performance dip that lasts about 2-6 months. In contrast, a team which learns Focusing skills first and moves to Delivering later will have two dips: a 1-4 month dip when they learn Focusing skills and another 2-6 month dip when they learn Delivering skills.

Agile teams’ performance changes in other ways, too. Agile teams focus on getting features completely done before moving on to the next feature. This is particularly true for Delivering teams, which build quality in from the beginning rather than fixing bugs at the end. This improves throughput and performance, but—ironically—it feels like a slow-down to people who are used to seeing multiple features in progress at once.

The net result is that stakeholders can be frustrated by the pace of Agile development, particularly in the first year, when they’re dealing with three hits all at once: a real delay from learning, a perceived delay from a focus on getting things done, and the cost of finishing pre-Agile work that was declared “done” without actually being done.

This frustration can lead to teams being redirected away from learning Agile, and solely focused on delivering software, before they’ve finished learning. This is counterproductive for everyone: teams will feel yanked around and frustrated, and the organization will have wasted the investments they’ve made so far. Before teams begin their Agile journey, make sure managers and stakeholders are on board with the first-year performance dip.

Your organization can trade money for time by hiring people to help your teams. This won’t eliminate the performance dip, but it will make it shorter and shallower. There’s a wide variety of help available: occasional mentoring; training; help with process design and implementation; and full-time (or near-full-time) coaching. The most effective help you can get is to hire experienced practitioners to coach each team full-time.

As you consider who to hire, ignore the myriad Agile certification schemes. Too many are money grabs. Most demonstrate nothing more than the ability to connect butt to chair for a few days.3 Some are associated with excellent training courses, but that’s due to the trainer, not the certification, so evaluate training courses independently of the certifications they tout. The same goes when hiring consultants and coaches. Ask your network for recommendations, sample publicly-available materials, and check references.

3Martin Fowler says, “For a certification to be useful, it needs a correlation with competence in the thing that it certifies. So if Alice has a certification in, say, clojure programming; there should be a high probability that Alice is a competent clojure programmer.” [Fowler 2011] In private communication, he went on to say that usually isn’t the case. “I certainly don’t know of any in the agile world.”

As you apply this book’s practices, you’re likely to run into problems and challenges specific to your situation. Be sure you have a mentor you can reach out to for questions. This doesn’t have to cost money; a respected colleague at a company who’s done it before, a local user group, or an online forum are all good options.

If there’s no time for learning...

You can make the performance dip less noticeable, at the cost of larger overall expense, by starting with just the Focusing zone and easing into Agile’s focus on getting things done.

If your organization won’t accept any perfomance dip at all, now isn’t a good time to invest in change. If there never seems to be a good time, that’s a big red flag. You’ll need to convince them to make time for change before you continue.

If there’s no budget for help...

With this book, the many free resources available online, and a dedication to learning, your teams can teach themselves everything they need to know. Outside help is, well, helpful, but it’s not required.

Choose or Create Agile Teams

I can’t overstate how important teams are in an Agile organization. Most organizations consider people their fundamental work-producing “resource.” In Agile, teams are the resource.

An Agile team is:

  • Cross-functional. The people on the team collectively have all the expertise needed for the team to fulfill its purpose.

  • Fully dedicated. Everybody on the team is assigned to the team 100%. Specialists can come help from time to time, but core team members are solely dedicated to their team.

  • Collaborative. Team members have friendly, collegial relationships and work closely together. Over time, the team “jells” into a high-performance whole that’s greater than the sum of its individual contributors.

  • Long-lived. A highly-productive “jelled” team is worth its weight in gold. It can take months for team members to figure out how to work most effectively together, so keep teams together for as long as possible.

The size and composition of each team depends on which fluency zones you’re pursuing. The “Whole Team” practice has the details, but briefly:

  • Focusing teams focus on achieving business results. They need people with the ability to put themselves in users’ and customers’ shoes to determine exactly what the software will do. If the team’s purpose is user-centric, that includes people with UI/UX skills. They also need a way to determine what to work on next. Although it’s best if the team includes people with the skill and authority to do this themselves, they can also work with someone from outside the team.

  • Delivering teams take responsibility for end-to-end delivery of their software. They need all skills required to build and deploy their product. Responsibilities that were previously handed off to other teams need to be brought into the team. This includes build management, data architecture and administration, testing, and operations.

  • Optimizing teams take responsibility for the broad business success of their product. They also take responsibility for coordinating with stakeholders and deciding product priorities. They need team members who have business, market, and product expertise.

You may already have teams that fit the bill. If you’re creating new Agile teams, use the following steps. Either way, remember to get teams’ buy-in, as described in the “Get Team Buy-In” section of the next chapter.

  1. Decide the purpose of each team. (See the “Establish a Learning-Friendly Purpose for Each Team” section.)

  2. Decide how many people will be on each team, based on how valuable the team’s purpose is, subject to the size limits described in the “Whole Team” practice.

  3. Determine which skills each team needs.

  4. Choose people that have the skills each team needs, are likely to work well together, and are willing to try Agile.

If you’re creating or reorganizing a lot of teams, consider using team self-selection. It’s surprisingly effective at creating highly-productive teams that are excited to work together. The book Creating Great Teams: How Self-Selection Lets People Excel [Mamoli and Mole 2015] describes how it works.

If you can’t dedicate people to their teams...

Agile depends on close collaboration, and that doesn’t work well if people aren’t available. Occasional outside responsibilities are fine, but if you can’t get dedicated team members, it’s unlikely Agile will work for you.

If you can’t create long-lived teams...

It’s wasteful to break up high-performing teams, but it won’t stop your teams from being Agile.

If team members don’t get along...

It’s normal for new teams to go through a rough patch while they figure out how to work together, so don’t worry if a team struggles at first. The team’s coach and manager can help mediate conflicts.

If a team has a history of interpersonal conflict, they might not be a good fit for Agile. If there’s just one person whose behavior causes conflicts, you might be able to solve the problem with individual mentoring or by moving them to a different team.

If you can’t get the business, customer, or user expertise you need...

Optimizing teams needs at least one team member with product management skills, but they don’t necessarily need a traditional product manager. Sometimes developers with a lot of company history know their product and its market better than anyone else. If that’s the case, you’re good to go.

If your teams aren’t pursuing Optimizing fluency, you don’t need a product manager directly on the team, but you still need someone with those skills to work closely with the team, and you still need team members who can represent customer and user perspectives. It’s crucial, and often overlooked.

[Coffin 2006] describes an experience with two nearly identical teams: one that didn’t include users’ perspectives and one that did. The team with no users took 15 months to produce a product with mediocre value:

The total cost of the project exceeded initial expectations and the application under delivered on the user’s functional expectations for the system... real business value was not delivered until the second and third [releases] and even then the new system was not perceived as valuable by its users because it required them to change while providing no significant benefit.

A team composed of many of the same developers, at the same company, using the same Agile process, later produced a product with compelling value in one-fifth the time.

The first production release was ready after 9 weeks of development... it surpassed scheduling and functional expectations, while still coming in on-budget... The application development team continued to deliver new releases to production approximately every six weeks thereafter. Every release was an exciting opportunity for the team of developers and users to provide value to the company and to improve the user’s ability to accomplish their jobs.

One of the primary reasons for this success was user involvement.

Many of the shortcomings of the [first] system stemmed from a breakdown in the collaborative atmosphere that was initially established. Had users been more involved throughout the project, the end result would have been a system that much more closely aligned with their actual needs. They would have had a greater sense of ownership and communication between the various groups would have been less tense.

...

The success of the [second] system caused many people in the organization to take note and embrace the lessons learned in this project... other project teams restructured their physical arrangements into a shared project room as the [second] team had done.

Business involvement makes a huge difference to team success. It’s one of the things that sets Agile apart from its predecessors. Make an extra effort to include business, customer, and user perspectives in your teams. If you don’t, the software they deliver is likely to disappoint.

If you can’t get all the developer skills you need...

Delivering fluency depends on bringing a variety of development skills, such as testing and operations, into the team. If that isn’t possible, your teams won’t be able to achieve Delivering fluency. That said, Delivering practices are still worth learning even if teams can’t use all of them. Over time, they can use their experience to make the case for bringing in the people or training they need.

Delegate Authority and Responsibility to Teams

Respect for people’s ability is central to the Agile philosophy.

Respect for people’s ability is central to the Agile philosophy, and nowhere is this more apparent than in Agile’s approach to authority and responsibility.

Top-notch execution lies in getting the details right, and no one understands the details better than the people who actually do the work... When equipped with necessary expertise and guided by a leader, they will make better technical decisions and better process decisions than anyone can make for them. [Poppendieck and Poppendieck 2003]

Mary and Tom Poppendieck

As organizations become more Agile, they entrust more and more decisions to cross-functional teams. You can see it in the fluency zones:

  • Focusing teams are responsible for their process and task planning.

  • Delivering teams are responsible for testing, deployment, and operations.

  • Optimizing teams are responsible for business results.

From an organizational investment perspective, this means:

  • Work is assigned to teams, not individuals. Teams decide for themselves how to break their work down into tasks, and who on the team will perform those tasks. You may need to change ticketing systems and other workflow processes to fit this approach. Doing so has implications for performance evaluations, which ties into the “Change Harmful HR Policies” section.

  • Teams decide their own processes. In particular, teams need to be free to use their own lightweight, tool-free approach to day-to-day planning and task tracking rather than being tied to a corporate tool. Management can put constraints on teams’ processes, but they must ensure each constraint has a clearly-articulated purpose. The “Scaling Agility” chapter explains further.

  • Focusing teams work with stakeholders to understand business needs and priorities. The organization needs to make sure teams have easy access to stakeholders or their representatives.

  • Delivering teams control their development, build, test, and release processes. Again, management can put constraints on those processes, such as mandating the use of a corporate release pipeline, but make sure teams have the ability to develop and release on their own without waiting for other teams.

  • Optimizing teams control their own budget and product plans. Management defines each team’s purpose, determines overall strategy, and sets the teams’ budgets. They also provide oversight in the form of reviewing business indicators. Within that framework, the organization needs to allow individual teams to decide for themselves how to achieve their purpose and spend their budget.

If work must be assigned to individuals...

If your organization isn’t comfortable with teams making their own task assignment decisions, your organization lacks the trust that Agile requires. You might be able to convince people to change their thinking by trying team-based work with a pilot Agile team, but proceed with caution. Command-and-control management styles are generally incompatible with Agile.

If it’s not a widespread issue, but just a few individual managers that have trouble letting go, see the “Change Team Management Style” section.

If tools don’t support team-based work...

If your company has existing work assignment systems that are difficult to change, a short-term solution is to create a “phantom” person for each team who receives the team assignments. Alternatively, team members can choose to treat individual assignments as team assignments.

Long term, it’s better to fix the tools.

If teams have to use a corporate tracking tool...

One of Agile teams’ biggest sources of leverage is the ability to improve and streamline their process. Corporate tracking tools, including so-called “Agile Lifecycle Management” tools, limit teams’ leverage. Like so many of the products jostling for space on the Agile bandwagon, these tools tend to miss the point of Agile so badly that they actually decrease teams’ agility.

Forcing teams to use a corporate tracking tool for their daily work will decrease their performance.

Forcing Agile teams to use a corporate tracking tool for their daily work will decrease their performance. If you don’t have a choice in the matter, a common solution is to maintain two tracking systems: a lightweight Agile approach, and the corporate tool. Every day, somebody on the team updates the corporate tool from the lightweight tracker.

You can reduce these costs by decreasing the amount of detail teams are required to put into the corporate tool. Ask if the tool really needs to be updated every day and for every task. Instead, see if teams can update it weekly or biweekly. The “Roadmaps” practice describes how.

If teams don’t have access to stakeholders...

Unlike waterfall processes, which use an up-front requirements and business analysis phase, Agile teams work with stakeholders throughout development to refine plans and gather feedback. Without access to stakeholders, they won’t build the right thing.

If a team can’t work with one or more stakeholder groups, make sure the team has access to someone who represents those groups’ interests. Choose this person carefully: the quality of the team’s product will depend on that person’s availability and ability to accurately represent stakeholders’ needs.

If Delivering teams don’t have control over their release processes...

You won’t see the full benefit of Delivering fluency until your teams have control over their release processes. That said, there’s enough value to Delivering practices that the zone is still worth pursuing. You can chip away at the problem over time.

If Optimizing teams don’t have control over their product plans and spending...

Optimizing teams need the ability to conduct experiments and adapt their plans, and that requires them to control their plans and spending. Without it, you won’t see the benefits of Optimizing fluency.

Change Team Management Style

With teams deciding their own process, making their own task assignments, and coordinating with stakeholders, team-level managers could think there’s no place for them in Agile. But that’s not remotely true. The job of the Agile team manager changes, but it’s no less important than in a pre-Agile team. See the “Ownership” chapter for details.

Talk with managers about their new role and provide training as needed. Make sure their managers’ expectations have changed to match.

If managers have trouble letting go...

Micromanagement is annoying, but isn’t a deal-breaker in the short term. It does inhibit learning, though, by taking decisions out of team members’ hands. Micromanagers will increase the time and cost required to reach fluency.4

4Thanks to George Dinwiddie for making this point.

Managers often micromanage when they don’t what else to do, or when they fear that there won’t be a place for them in an Agile environment. Reassure managers that they still have a role by showing them what that role looks like. Training or a good Agile coach can help.

Choose Agile Coaches

Each team needs a coach to help them learn how to be an effective Agile team. The “Whole Team” practice has the details (see the “Coaching Skills” section), but briefly:

  • Every team needs someone who can help them learn how to be an effective, jelled team.

  • Focusing teams need someone who can teach them the planning practices described in Part II.

  • Delivering teams need someone who can teach them the technical practices described in Part III.

  • Optimizing teams need someone who can teach them the business development practices described in Part IV.

Depending on which zones your teams are developing, you might need more than one coach for each team. Each coach can work with one or two teams.

If you can’t hire the coaches you need...

You can grow your own Agile coaches. Choose senior practitioners that team members respect and trust—if they’re not immediately obvious, ask your teams for recommendations—and ask them to take on the challenge. This book has everything they need to get started. Player-coaches that are fully dedicated to a single team are your best choice.

Create Team Rooms

Agile teams are highly collaborative and communicate constantly. In order for that communication to be effective, they need a team room designed for their needs. For in-person teams, it will be a physical space. For remote teams, it will be virtual. The “Team Room” practice has the details.

For in-person teams, creating physical team rooms can be one of the most expensive investments you’ll make. It’s also one of the most valuable; as the “Team Room” practice discusses, physical team rooms act as performance multipliers.

Teams new to Agile underestimate how much they’ll enjoy collaborating.

When your teams are just getting started, though, you may not yet know what sort of team rooms they need, or even if Agile is a good choice long-term. Your teams probably won’t either. Teams new to Agile underestimate how much they’ll enjoy collaborating and overestimate their desire for privacy.

So it’s okay to hedge your bets on physical workspaces. Set aside the budget for it—you’ll need good team rooms eventually, if you stick with Agile—but in the short term, you can commandeer a large conference room or part of an open-plan office for each team.

Whatever you decide to do, start working on it early. Physical team rooms take a long time to arrange.

If you can’t create a physical team room...

Remote teams can use virtual team rooms.

In-person teams can use virtual team rooms, too, but I strongly recommend against this. They will experience the worst of both worlds: the inflexibility and commute of in-person work, combined with the communication challenges of remote work.

Physical team rooms are expensive, but worth it. Do note that they don’t have to be literal rooms with walls. (See the “Team Room” practice for details.) If you’re not sure if the expense is justified, set up in a conference room for at least three months to evaluate.

Establish a Learning-Friendly Purpose for Each Team

Every team has a purpose: their place in the organization’s big-picture strategy. (See the “Purpose” practice.) When a team is learning Agile for the first time, it’s important to choose a purpose that will help them learn. Practically speaking, this means three things:

  • A purpose that’s valuable, but not time-sensitive. If the team’s under a lot of time pressure, they’ll have trouble learning. They’ll default to what’s worked for them in the past rather than taking time to learn new ideas.

  • A purpose that’s self-contained. The more the team depends on other teams, the more coordination challenges they’re likely to face. Some coordination challenges are to be expected, but too many will distract the team from learning.

  • A green-field (brand-new) codebase. Teams learning Delivering practices have a lot to learn, and it’s easier to do so with green-field code. That said, they’ll need to learn how to work with existing code eventually. Teams that have an experienced Delivering coach can ignore this requirement, if the coach agrees. So can teams that aren’t using Delivering practices.

If you have multiple teams, you’ll need to make sure their purposes align. See the “Scaling Agility” chapter for details.

If there’s an important deadline...

Each team needs plenty of time to learn. If the deadline leaves lots of room to maneuver, you’re okay. If not, it’s usually better to delay trying Agile until after the deadline is met, or to choose different teams.

If there’s no valuable green-field work to do...

It’s more important for teams to do valuable work than to have a green-field codebase. Without an experienced coach, though, teams learning Delivering practices for the first time are likely to have difficulty with pre-existing code. Expect a longer performance dip, more time needed to reach fluency, and more frustration from programmers on the team.

Replace Waterfall Governance Assumptions

For Agile to work well, waterfall governance policies must be changed.

Governance is the way work is approved, tracked, and managed at a high level. Most organizations’ governance policies assume a waterfall development approach. For Agile to work well, those policies must be changed.

Early software development revolved around projects, which had clear start and end dates. When the software shipped, the project was over. When more work was needed, a new project would be started.

Modern software development doesn’t work that way, of course. Software today involves ongoing development, maintenance, and operations. Software is more like a product than a project. Some level of ongoing work is required until the product is retired. But many organizations still fund software development on a project-by-project basis.

Project-based accounting results in a whole series of complications. Projects typically require a fixed budget to be approved in advance. But if you need a budget in advance, you need to commit to a plan in advance. To commit to a plan in advance, you need to know your requirements in advance. To ensure you don’t exceed your budget, you need a way of monitoring conformance to the plan. In other words, you need a predictive process. But Agile is adaptive, not predictive.

Instead of using project-based governance, fund teams on an ongoing “business as usual” basis. Your biggest expense will be people (usually), so budget for a team of X people, where the cost of X is less than the value they’re expected to produce. Track progress through teams’ demos (see the “Stakeholder Demos” practice) and expect each team to ship early and often. Look for a steady stream of value, whether that’s revenue or something else. Manage the team at a strategic level: monitor the value they create and adjust their purpose, budget, and team size accordingly. When the team stops providing enough value, shut down their current purpose, or put it on the back burner, and give them a new purpose to fulfill.

This is simplified, but the point holds: for Agile teams, allocate an ongoing budget based on your estimate of value, not cost. Expect your Agile teams to deliver value early and often, measure the actual value they create relative to your estimates, and adjust your plans and costs to match.

If waterfall governance is required...

It’s wasteful, and will detract from your teams’ agility, but you can adhere to waterfall governance policies if you need to. It’s okay for a few pilot teams, but switch to Agile governance before spreading Agile further.

The most common governance demand is to produce a fixed plan and budget in advance. The simplest way to meet this demand is to use whatever approach you use now, then start the Agile part of the process after you’ve gone through project approval. Alternatively, for teams that are fluent in both the Focusing and Delivering zones, you can allocate 4-8 weeks for “planning,” start working normally, and use the “Forecasting” practice to dial in a high-quality roadmap.

Other up-front documentation, such as a requirements analysis document or design document, can also be done using existing approaches prior to starting the Agile part of your work. Remaining compliance work can usually be scheduled alongside your Agile work, with stories (see the “Stories” practice), like any other request.

Waterfall governance is incompatible with Optimizing fluency, which relies on adaptive planning. If you’re required to adhere to waterfall governance policies, limit your aspirations to the Focusing and Delivering zones.

If you’re a software development vendor...

If you provide outsourced software services, or other sorts of contract-based software development, you can still be Agile. Your approach will depend on the type of software contract you use.

Time and materials (T&M) contracts are the most compatible with Agile. In a T&M contract, the client simply pays the vendor for work performed. It’s similar to the "business as usual" funding Agile teams prefer.

Predictive T&M contracts put all the risk of incorrect predictions on the client, because if the work takes longer than expected, the client has to keep paying or be left with nothing. Agile T&M contracts can mitigate these risks by specifying regular demonstrations of work in progress, at least once per month delivery of working software, and the opportunity to change or cancel the work after every delivery. This allows the client to use Agile governance of the style described previously.

Fixed-price contracts, in contrast, puts the risk of incorrect predictions on the vendor. In a fixed-price contract, the project requirements are defined in advance. The vendor quotes a price based on those requirements, and if development takes longer than predicted, the vendor eats the extra cost. Vendors typically mitigate this risk by charging high change fees. They’ll often quote a low price in order to win the bid, then make their profit from the inevitable change requests.

Given the inevitability of changes in software development, fixed-price software contracts create an adversarial relationship between client and vendor that’s at odds with Agile’s philosophy of customer collaboration and responsiveness to change. That said, Agile teams can perform fixed-price work. Start with whatever estimating and bidding process you use currently, then use your Agile teams for the development and delivery portion of the work.

Contracts that depend on up-front requirements are incompatible with Optimizing fluency, but they’re fine for the Focusing and Delivering zones.

Change Harmful HR Policies

Agile is a team sport, and despite paying lip service to teamwork, many companies have policies that unintentionally discourage it. Any policy that puts people in competition with each other is going to make Agile more difficult. A particularly destructive example is stack ranking, in which team members are evaluated relative to each other, with the top of the stack getting promotions and the bottom getting fired, regardless of their actual performance.

A related issue is managers who only value tangible output. On an Agile team, there are many ways to contribute to success, such as the person who doesn’t write a lot of code, but spends a lot of time reproducing bugs, or the person who works in the background to improve communication.

Rewarding individuals instead of teams makes this worse. It sends a message that assisting others is less important than pursuing your own goals. At its worst, emphasis on individual work turns into hero culture, where talented individuals are rewarded for working long hours in isolation, often to the detriment of overall team performance.

Organizations can also develop blame culture, which responds to mistakes by punishing culprits. The Agile mind-set, in contrast, treats mistakes as a learning opportunity. For example, a non-Agile organization might fire a programmer for accidentally deleting a crucial production database. An Agile organization would instead ask, “what checks and balances can we put in place to prevent accidental database deletion, and how can we make it easier to recover from these sorts of mistakes?”

These sorts of cultural issues are often reflected in HR policies about promotion and rewards. If people’s careers depend on making themselves look good, regardless of their actual effect on team performance, your teams are likely to have trouble with Agile’s emphasis on collaborative work.

You won’t be able to change your organization’s culture overnight, but you can work on changing HR policies, and managers can change the way they treat their teams. This will take time, so start working on it early. You’re likely to need the backing of senior management.

It helps to look for creative ways to apply existing policies, rather than removing policies altogether, which can be more difficult. Remember, too, that managers often have discretion in how they apply policies, so if you hear that something “can’t be done,” it may be a manager that you need to convince, not HR.

If HR policies are set in stone...

If you can’t change bad HR policies, managers will have to shield their teams from them. Make sure they’re on board with Agile and are savvy about navigating corporate bureaucracy.

If you have a lot of teams, limit your Agile pilot to teams with sharp managers. Use their experiences to motivate the policy changes you need.

Address Security Concerns

This investment usually isn’t an issue, but when it is a problem, it will stop you cold. So check into it, especially if you’re in a security-sensitive industry.

The issue is that colocated teams who use practices such as the “Pair Programming” practice and the “Mob Programming” practice work together at the same computer. This can be worrying from a security perspective, because the person who logged in to the computer isn’t always the one who’s at the keyboard. In fact, the person who logged in might even step away for a moment to go to the bathroom or have a side conversation. Because people switch who is at the computer frequently—as often as every few minutes—logging out and back in again every time a new person is at the keyboard isn’t feasible.

If your teams will use those practices, run them by your company’s security team and work with them to resolve their concerns. You can usually find a creative way to support Agile work while also remaining secure. One common approach is to create a locked-down, shared development account. Some companies combine this with dedicated development workstations or shared server-based VMs. Email and other individual work takes place on individually-assigned laptops.

A related issue is traceability. Some companies require that every commit be traceable to its original authors. You can meet this requirement by adding the authors’ initials or email addresses to the commit comment. Git has a convention of adding Co-authored-by lines to the end of the commit message.5

5Documented at https://git.wiki.kernel.org/index.php/CommitMessageConventions. Thanks to Jay Bazuzi for bringing this to my attention.

Some companies require that all code be reviewed prior to release. Pairing and mobbing meet this requirement, but you may need to modify tooling to allow code to be released without a separate review phase. If removing the requirement entirely isn’t an option, you might be able to modify the tool to skip commits that have co-authors.

If there’s no flexibility around security requirements...

You can require that the person who logged in stays at their computer. If they need to step away for a moment, either they switch logins, or work stops until they’re back. This introduces more friction than you might expect, so prefer solutions that allow work to continue.

Teams can also use tools designed for collaborative remote work rather than working at the same computer. This introduces a lot more friction than the other options, even when team members are sitting side-by-side, so I don’t recommend it unless your team is already remote.

If you’re required to have a separate code review step...

Code written by a pair or mob has already been peer reviewed, so teams can rubber-stamp reviews of that code. It adds friction, though, so it’s best to remove this requirement prior to spreading Agile widely.

Go Shopping

Whether they’re in-person or remote, your teams will need equipment, tools, and supplies to make collaboration more effective.

For in-person teams, provide following equipment and supplies. Although some of these can be replaced with electronic tools, invest in the physical equipment. It’s not very expensive and it will allow your team to take advantage of the strengths of face-to-face collaboration.

  • Two big magnetic whiteboards6 for planning purposes. I like to use a single double-sided, six-foot wide magnetic whiteboard on wheels. It makes it easy for the team to roll their plans into meeting rooms. It needs to be magnetic so you can stick cards to it easily.

  • Lots of additional whiteboard space, preferably magnetic, for discussions and charts.

  • A large plastic perpetual calendar (three months or more) for marking important dates.

  • Sound-damping partitions to define the team’s workspace and prevent noise pollution, or an enclosed team room.

  • Stools or other ways to easily sit with someone temporarily.

  • Notepads or handheld whiteboards for sketching ideas while seated.

  • Miscellaneous toys and conversation pieces to inspire discussion and interaction among team members.

  • Flip chart paper, one or two flip-chart easels, and a way to attach flip chart pages to walls (such as blue tape, poster tack, or T-pins).

  • Index cards in a variety of colors (at least 2,000 of each color). Make sure everyone on the team can distinguish the colors.7

  • Sticky notes in a variety of colors, sizes, and shapes.

  • Pencils, pens, and water-based felt-tip pens for writing on cards and sticky notes. Avoid permanent markers such as Sharpies; they have a strong odor and invariably get used on a whiteboard.8

  • Dry-erase markers for whiteboards, water-based flip-chart markers for flip-charts, and wet-erase markers for semi-permanent whiteboard notes and the perpetual calendar. Avoid scented markers.

  • Magnetic push-pins for attaching index cards and documents to whiteboards.

  • A copy of this book and any other useful reference material.

6Technically, a “magnetic whiteboard” isn’t actually magnetic, but has a steel backing so that magnets stick to it.

7One out of eight men have some form of color-blindness.

8Pro tip: you can remove permanent marker from whiteboards by writing over it with a whiteboard marker, then immediately erasing.

In-person teams using pair programming also need pairing stations (see the “Pair Programming” practice for details):

  • Wide desks suitable for side-by-side work. Some teams prefer a mix of standing and sitting desks, or adjustable-height desks.

  • One development-grade computer at each pairing station.

  • Two keyboards and mice at each station. Some people prefer to use their own keyboard and mouse; in that case, make sure each computer’s USB ports are easily accessible, because they’ll be moving between pairing stations multiple times per day.

  • At least two monitors at each station.

In-person teams using mob programming need a mobbing station (see the “Mob Programming” practice for details):

  • Desks with enough seats for every member of the team, plus a few guests, with room for people to easily switch seats.

  • A “driver’s seat,” easily accessible, with a mouse, keyboard, and development-grade computer.

  • A “navigator’s seat” at the same desk as the driver’s seat, or close enough for the driver and navigator to easily talk to each other.

  • At least one monitor, typically 60" diagonal or more, that’s large enough to be seen clearly from all seats. 4K TVs often work well for this purpose. Make sure to accommodate everyone’s vision needs.

Remote teams need an electronic version of the team workspace:

  • Videoconferencing software, such as Zoom, for real-time conversation.

  • Messaging software, such as Slack, for asynchronous conversation.

  • Collaborative workspace software, such as Miro or Mural, for free-form collaboration.

  • Collaborative versions of task-specific tools, where possible, such as Figma for UX and UI design.

  • Inexpensive tablets for collaborative whiteboard sketches.

  • An additional monitor or tablet for videoconferencing, so people can see each other and work at the same time.

  • For Delivering teams, collaborative programming tools, such as Tuple or Visual Studio Live Share, that support pairing or mobbing (see the “Pair Programming” practice and the “Mob Programming” practice for details).

In all cases, avoid purchasing new licenses for Agile Lifecycle Management software or other tracking software unless the team explicitly requests it. Even then, wait until the team has tried the lightweight planning practices described in Part II.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Chapter 4: Choose Your Agility

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Choose Your Agility

Agile includes a lot of ideas. Which ones should you actually use? The answer depends on what you want to achieve, and what your organization is willing to invest in return. To decide, use the Agile Fluency Model.1

1“Agile Fluency” is a registered trademark of Agile Fluency Project LLC, founded by Diana Larsen and James Shore.

The Agile Fluency Model

In 2014, I partnered with Diana Larsen to analyze why companies see such different results from their Agile teams. We had both worked with Agile teams from the beginning, and, over the years, we noticed that they tend to cluster in different “zones” of capability. We captured these observations in the Agile Fluency Model. A simplified view is shown in the “Agile Fluency Model” figure. [Shore and Larsen 2018]

A picture of the Agile Fluency Model. It shows a path from “Pre-Agile” through a shift in team culture to “Focusing,” followed by a shift in team skills to “Delivering,” then a shift in organizational structure to “Optimizing,” and finally a shift in organizational culture to “Strengthening.” The path continues and fades away, as if there are additional zones yet to be discovered.

Figure 1. A simplified view of the Agile Fluency Model

Although this simplified figure shows a straightforward path from one zone to the next, reality is messier: the zones are actually clusters of skills, and they all develop in parallel. A team has proficiency in a zone when they’re able to apply all of its skills. They have fluency when they can do so without conscious effort. Teams can achieve fluency in any order, but the progression shown in the figure is most common.

Each zone is associated with a set of benefits. To reap those benefits, a team needs to be fluent in that zone. But achieving fluency isn’t just something do on their own. Your organization also has to invest in supporting them. That means more than paying lip service to Agile ideas. They have to make actual, meaningful changes that cost time, money, and political capital.

In other words, the results you get from your Agile teams depend on how well your company buys in to Agile ideas. When a company fails to achieve the results they want from Agile, it’s usually because they didn’t make the investments they needed to. Often, they don’t even realize they had to participate. So they don’t act, and fail by default.

Make a conscious choice to invest in agility.

Make a conscious choice to invest in agility. Consider each zone carefully. Each has costs; each has benefits. Choose the ones that have the right cost/benefit tradeoffs for your situation.

You probably won’t be able to convince your company to invest in every zone. That’s okay. In contrast to maturity models such as the Capability Maturity Model Integration (CMMI), the fluency model doesn’t show a progression from low skill to high skill. Instead, it shows multiple investment/benefit choices. Although the diagram shows the most common progression, each zone may be chosen independently. Each has value on its own.

Focusing Zone

The Focusing zone is about Agile fundamentals: focusing on business results; working as a team; taking ownership. Teams that are fluent in this zone see the improvements in value described in the “Organizational Success” section of the “Why Agile” chapter. They focus development on their team’s core purpose, release their most valuable features first, and change direction in response to changing business needs.

They also see some of the improvements to personal success described in the “Personal Success” section of that chapter. Specifically, stakeholders like their improved visibility into the team’s work and their ability to influence teams’ plans.

To achieve these results, Focusing teams regularly review and change their plans. They work as a tightly-knit team to define their own work processes and self-organize around completing their work. All the while, they’re Focusing on the next piece of value for their organization.

For most teams and organizations, this requires a shift in how they think about teams. Pre-Agile organizations make plans in advance, ask teams for estimates, and expect reports about how work is progressing relative to those estimates. Focusing teams revise their plans frequently—at least every month—and demonstrate progress by showing what they’ve completed.

Pre-Agile organizations break their plans into tasks, assign those tasks to individuals on the team, and judge individuals based on how well they complete their tasks. Focusing teams do their own task breakdowns, decide for themselves who will work on each task, and expect to be judged on their ability to create value as a team.

These changes correspond to the essence of Agile: Agile is adaptive rather than predictive, and people-oriented rather than process-oriented.

Can your organization buy into this core philosophy? Again, lip service isn’t enough. To succeed, they’ll need to make concrete investments in the form of changes to team structure, management, and work environment. (See the “Invest in Agility” chapter for details.) It’s a “good news, bad news” situation: The bad news is that, when the rubber meets the road, some organizations won’t be willing to invest. The good news is that, if they refuse, you’ve discovered early on that they’re not really on board with Agile ideas. You just saved yourself years of frustration and heartache chasing Cargo Cult Agile.

If you are able to get buy-in, Focusing fluency will take each team about 2-6 months of dedicated effort to achieve. With proper support, they’ll exceed their prior levels of performance within 1-4 months.2 Part II has the practices they’ll need.

2The time frames in this chapter are all ballpark figures based on my experience. Your experience may be different.

Despite its benefits to organizational and personal success, the Focusing zone won’t improve technical success, and the cost savings described in the “Organizational Success” section of the “Why Agile” chapter aren’t likely to appear, either. In fact, technical quality could get worse. Traditional approaches to design rely on careful up-front work based on firm plans. That doesn’t mesh well with Agile’s emphasis on flexible, frequently changing plans. As a result, Focusing teams often end up with an ad-hoc approach to design, which isn’t sustainable.

Poor technical quality can be frustrating for team members, lowering their feelings of personal success. It usually prevents the team from making reliable forecasts and commitments, which can be frustrating for stakeholders.

To improve technical success and make useful forecasts, you’ll also need fluency in the Delivering zone.

Delivering Zone

Agile teams may change their plans at any time. For most teams, this slowly degrades the quality of their code. They gradually lose their ability to make cost-effective changes. Eventually, they say they need to throw away the software and rewrite—an expensive and wasteful proposition.

Delivering teams prevent this problem by focusing on technical agility. They design their code so that it can absorb frequent changes. They keep code quality high, so they don’t waste time hunting bugs. They refine their production lifecycle so releases are painless and operations are manageable. They’re capable of Delivering reliable, low-defect software whenever it makes the most business sense.

This results in the technical success described in the “Technical Success” section of the “Why Agile” chapter, the cost reductions described in the “Organizational Success” section, and the team member success factors described in the “Personal Success” section. Combined with Focusing fluency, that’s all the success factors described in that chapter.

Achieving these results requires a substantial investment in team members’ skills, as well as structural changes to integrate people with skills such as testing and operations into each team. (See the “Invest in Agility” chapter for details.)

If your company makes these investments, Delivering fluency will take each team 3-24 months to develop, and you’ll see improved performance within 2-6 months. Part III has the practices. The exact amount of time each team needs will depends on the quality of their existing code and how much coaching they receive.

Optimizing Zone

Most companies would be satisfied with Focusing and Delivering fluency. But Agile imagines more. In its full glory, Agile is a world in which teams twirl and dance in response to changing market conditions. They experiment and learn; develop new markets; outmaneuver the competition.

Optimizing teams achieve this level of agility. They understand what their market wants, what your business needs, and how to bridge the two. Or, as in a startup environment, they know what they need to learn and how to go about learning it. They’re constantly Optimizing their product plans to achieve the most value possible.

This requires a shift in organizational structure. Creating optimal plans requires constant attention by people with deep business and product expertise, and that means having product and market experts join development teams full-time. It also means giving those teams full responsibility for their product budgets and plans.

These structural changes require high-level permission in the organization. It can be difficult to obtain. Teams typically need to spend at least a year building trust via Delivering fluency before they get permission for these investments. Once that happens, Optimizing fluency takes another 3-9 months to develop, although you’ll see improvements within 1-3 months. But even then, Optimizing is a never-ending process of experimentation, learning, and discovery. Part IV describes how to begin.

Strengthening Zone

There’s one final zone in the Agile Fluency Model. It’s largely speculative: a possible future for Agile. It’s also only appropriate for organizations on the bleeding edge of management theory and practice. That puts it out of scope for this book. Briefly, the Strengthening zone involves distilling teams’ collective insights and channeling them into organizational improvements. If you’d like to learn more, start with [Shore and Larsen 2018].

Choose Your Zones

Which Agile ideas should your teams use? It depends on which zones your organization can support. In a vacuum, Focusing, Delivering, and Optimizing, all together, is your best choice. They combination of all three provides the best results and purest realization of Agile ideas.

But choosing all three zones also takes the most investment. If you can’t justify those investments, you’re likely to have trouble getting the support you need. And without sufficient investment, your teams will have trouble reaching fluency. You’ll incur the costs of learning without getting all the benefits. You might even see worse results than now.

In other words, only choose the zones your company both needs and is willing to pay for.

So, which zones should you choose?

  • Every Agile team needs Focusing fluency. It’s fundamental. If your company can’t at least invest in Focusing fluency, Agile isn’t a good fit.3

  • Delivering fluency decreases costs and increases development speed. It takes time to learn, but you’ll see benefits within 2-6 months, and it will pay for itself within a year. Without it, your code will eventually succumb to technical debt. That makes the Delivering zone a no-brainer for most teams. That said, some organizations aren’t ready to make the big investments in learning and code quality that the Delivering zone requires. It’s okay to start with Focusing fluency first, demonstrate success, and then use that to make the case for further investment.

  • Optimizing fluency is where Agile shines brightest. It’s also a lot to ask. For most organizations, it’s best to build trust by demonstrating fluency in the Delivering zone first, then gradually take on more responsibility. But if your organization already has a culture of delegating decision-making authority to cross-functional teams, as is often seen in startups, Optimizing fluency will give you great results.

3Some teams do pursue the other zones on their own. But this book uses Focusing fluency as a baseline.

If you aren’t sure, start with the Focusing and Delivering zones.

Whichever zones you choose, invest in learning all their practices simultaneously. The techniques in the later zones make the earlier zones work better, so you’re better off adopting them together rather than one at a time. But if you can’t invest in every zone you want, that’s okay. It takes longer, but as long as you can at least invest in Focusing fluency, you can build up to the other zones over time.

Once you’ve chosen your zones, it’s time to consider your organization’s investments in more detail. We’ll study them in the next chapter.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Chapter 3: How to Be Agile

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

How to Be Agile

How do you get from Agile’s conglomeration of ideas to actual, functioning Agile teams?

Practice. Lots and lots of practice.

Practicing Agile

Every team has a way of working—a process, or method—that they follow, even if it isn’t formally written down. The method reflects an underlying philosophy, although that philosophy is rarely articulated, and isn’t necessarily self-consistent.

To be Agile, you need to change your process to reflect the Agile philosophy.

To be Agile, you need to change your process to reflect the Agile philosophy. It’s both easier and harder than it sounds. It’s easy because, in most cases, you can start with one of the many off-the-shelf Agile methods, such as the one in this book. It’s hard because you need to change your way of working, and that involves changing a lot of habits.

The Agile community calls those habits practices. They’re things like planning sessions, automated builds, and stakeholder demos. Most have been around for decades. Agile methods combine them in unique ways, accentuating those parts that support the Agile philosophy, discarding the rest, and mixing in a few new ideas. The result is a lean, powerful, self-reinforcing package.

When you decide to follow an Agile method, it’s tempting to only use the practices that are similar to what you’re already doing and ignore the ones that are different. Avoid this temptation! The practices that are the least familiar are often the ones that involve the biggest change in philosophy. They’re the ones that your teams need the most, if they’re really going to be Agile.

In addition, Agile practices often perform double- and triple-duty, solving multiple software development problems simultaneously and supporting each other in clever and surprising ways. You won’t truly understand how it all works until you’ve seen it in action for a while.

So, although it’s tempting to customize your Agile method from the beginning, it’s best to start out with a by-the-book approach. Put it into practice—the whole thing—and spend several months refining your usage and understanding why it works. Then customize. Choose one of the rough edges, make an educated guess about how to improve, observe what happens, and repeat.

The Road to Mastery

The core thesis of this book is that mastering the art of Agile development requires real-world experience using a specific, well-defined Agile method. The majority of this book—parts two through four—is dedicated to that purpose. It’s a curated set of Agile practices that have been proven in the real world.1

1The practices in this book are primarily based on Extreme Programming, but I’ve also taken inspiration from a wide corpus of Agile ideas. The main influences are Scrum, Kanban, Lean Software Development, the DevOps movement, and Lean Startup.

To master the art of Agile development—or simply to use Agile practices to be more successful—follow these steps:

  1. Choose an Agile method, such as the one in this book. Use as many of its practices as you can. Agile practices are self-reinforcing, so it works best when you use them all together.

  2. Apply the practices rigorously and consistently. If a practice doesn’t work, try following the method more closely. Teams new to Agile often underapply the practices. Expect to take two or three months to start feeling comfortable with the practices and another two to six months for them to become second nature.

  3. As you become confident you’re applying the practices correctly—again, give it several months—start experimenting with changes. The practices in this book each include a discussion of why the practice works and how it can be changed. Every time you make a change, observe what happens and make further improvements.

  4. There is no last step. Agile software development is an ongoing process of learning and improvement. Never stop practicing, experimenting, and evolving.

The “Road to Mastery” figure illustrates the process. First, follow the rules; then break the rules; and finally, leave the rules behind.2

2This progression was inspired by Alistair Cockburn’s discussions of Shu-Ha-Ri, such as in [Cockburn 2001] (pp. 17-18).

A diagram with three sections, labelled “Follow the Rules (Apprentice),” “Break the Rules (Journeyman),” and “Ignore the Rules (Master).” Each section has double-headed arrows indicated that it’s possible to travel between each section. In the “Follow the Rules” section, a small flow chart starts by saying “Pick a method,” then shows an endless loop from “Follow it thoughtfully and rigorously” to “Reflect on your experiences” to “Study the method further” (and then back to “Follow it thoughtfully and rigorously”). In the “Break the Rules” section, a flow chart shows an endless loop from “Observe results. What could be better?

Figure 1. The road to mastery

How to Begin

Your first steps depend on what you want to accomplish. Are you joining an existing team, launching new teams, or improving the teams you already have?

Joining an Agile Team

If you’re planning to join an existing Agile team, or just want to know more about how Agile works in practice, you can skip ahead to parts 2-4. Each part starts with a “day in the life” story that describes what Agile can look like. Every Agile team is different, so the team you join won’t be exactly the same, but the material in parts 2-4 will give you an idea of what to expect.

After reading the stories, skip around to the practices that interest you. Each is written to be a standalone reference. The “Practices by Another Name” table can help you map the terminology your team uses to the terms used in this book. The “Practices by Skill Set” table will help you understand which practices are relevant to your skills.

Launching Agile Teams

If you’re helping your organization create Agile teams, or you want to convince them to do so, the remaining chapters in Part I will help you get started. Use the following checklists to stay organized.

First, make sure Agile is a good fit for your company:

  • Choose an approach to Agile that your organization can support. (See the “Choose Your Agility” chapter.)

  • Determine which investments your organization will need to make. (See the “Invest in Agility” chapter.)

  • Get management buy-in to making those investments. (See the “Invest in Change” chapter.)

Next, create an environment where Agile will thrive:

  • Work with management to decide on an initial “best guess” purpose for each team. (See the “Establish a Learning-Friendly Purpose for Each Team” section. If you have multiple teams, see the “Scaling Agility” chapter.)

  • Determine who will be on each team and who their coaches will be. (See the “Choose or Create Agile Teams” section and the “Choose Agile Coaches” section.)

  • Get team member and stakeholder buy-in. (See the “Invest in Change” chapter.)

  • Make the investments. (See the “Invest in Agility” chapter.)

In the weeks leading up to a team trying Agile:

  • Identify at least one person who will act as the team’s product manager. (See the “Whole Team” practice.)

  • Have the team’s product manager meet with the team’s executive sponsor and key stakeholders to create a draft purpose. (See the “Purpose” practice.)

  • Confirm that the team has everyone they need to fulfill their purpose. (See the “Whole Team” practice.)

  • Ensure the team has a physical or virtual team room. (See the “Team Room” practice.)

  • Schedule and conduct the team’s chartering session, with attention to logistics such as venue, travel, facilitation, and attendance. (See the “Planning Your Chartering Session” sidebar, the “Purpose” practice, the “Context” practice, and the “Alignment” practice.)

  • Ask the team to review their new practices. Provide copies of this book for people to study on their own, suggest that they try some practices in their current work, and consider providing training for practices that seem challenging. (The practices are described in parts 2-4. See the “Practices by Skill Set” table to cross-reference people’s skills to the practices they need to know.)

When a team is ready to begin, they’ll take a deep breath and start applying the practices! Their first activity will be to plan their first week. See the “Task Planning” practice for details.

Improving Existing Agile Teams

If you’re interested in fine-tuning your team’s existing approach, skip ahead to parts 2-4 and read the practices that interest you.

If you want to make bigger improvements, continue reading with the next chapter. The process is similar to launching new Agile teams: Choose the zones that work for your organization, get people on board, make investments, and follow the road to mastery.

If Agile isn’t working for your organization, check out the “Troubleshooting Guide” sidebar, and consider “rebooting” your teams by following the “Launching Agile Teams” checklists.

Applying Individual Agile Practices

Agile works best when you go all in, but if that’s not an option, you may be able to add some Agile practices to your existing process. These are good places to start:

  • Daily planning. If you struggle with frequent interruptions, try adopting day-long iterations (see the “Task Planning” practice). Use the planning game (see the “The Planning Game” practice) and your team’s measured capacity (see the “Capacity” practice) to conduct a joint planning session at the beginning of each day, then defer all interruptions until the next day’s planning meeting. Be sure to have people estimate their own tasks.

  • Iterations. If you aren’t interrupted frequently, but you still want to improve your planning, try using weekly iterations (see the “Task Planning” practice). In this case, you may also benefit from daily stand-up meetings (see the “Stand Up Meetings” practice) and regular stakeholder demos (see the “Stakeholder Demos” practice). As time goes on, consider using index cards for planning and a big chart to show upcoming work, as described in the “Visual Planning” practice.

  • Retrospectives. Frequent retrospectives (see the “Retrospectives” practice) are an excellent way for your team to adapt and improve its process. The other practices in the “Improvement” chapter may be helpful, too.

  • Ten-minute build. A fast, automated build will make a big difference to your quality of life, and it will open up improvements for other improvements as well. See the “Zero Friction” practice for more.

  • Continuous integration. Continuous integration—the practice, not the tool—not only decreases integration problems, it also drives improvements to your build and tests. See the “Continuous Integration” practice for details.

  • Test-driven development. Although test-driven development (see the “Test-Driven Development” practice) isn’t as easy to adopt as the other practices, it’s very powerful. Test-driven development is the basis for reducing bugs, increasing development speed, improving your ability to refactor, and decreasing technical debt. It can take some time to master, so be patient.

Other practices in parts 2-4 could be useful, too. Agile practices have a lot of dependencies on each other, so be sure to pay attention to the “Allies” blocks and the “Prerequisites” section of each practice.

Don’t be disappointed if you have trouble applying individual practices. It’s faster and easier to choose a coherent subset and go all in. That’s the topic of the next chapter.

Table 1. Practices by another name

Name You HeardName in This BookFound In
40-Hour WeekEnergized WorkThe “Energized Work” practice
BacklogRelease PlanThe “Visual Planning” practice
Backlog ItemsStoriesThe “Stories” practice
Daily ScrumStand-Up MeetingsThe “Stand-Up Meetings” practice
Daily Stand-UpStand-Up MeetingsThe “Stand-Up Meetings” practice
Definition of Done"Done Done"The “Done Done” practice
Ensemble ProgrammingMob ProgrammingThe “Mob Programming” practice
Iteration DemoStakeholder DemoThe “Stakeholder Demos” practice
Post-MortemsRetrospectivesThe “Retrospectives” practice
Product BacklogRelease PlanThe “Visual Planning” practice
Product OwnerProduct ManagerThe “Whole Team” practice
Program BacklogRelease PlanThe “Visual Planning” practice
Scrum MasterCoachThe “Whole Team” practice
Scrum TeamWhole TeamThe “Whole Team” practice
Sit TogetherTeam RoomThe “Team Room” practice
Sprint BacklogIteration PlanThe “Task Planning” practice
Sprint PlanningThe Planning GameThe “The Planning Game” practice
Sprint ReviewStakeholder DemoThe “Stakeholder Demos” practice
SprintsIterationsThe “Task Planning” practice
Sustainable PaceEnergized WorkThe “Energized Work” practice
Test-First ProgrammingTest-Driven DevelopmentThe “Test-Driven Development” practice
User StoriesStoriesThe “Stories” practice
VisionPurposeThe “Purpose” practice

Table 2. Practices by skill set

PracticeCustomer SkillsDevelopment SkillsCoaching Skills
Teamwork
Whole Team
Team Room
Safety
Purpose
Context
Alignment
Energized Work
Planning
Adaptive Planning
Visual Planning
The Planning Game
Stories
Incremental Requirements
Real Customer Involvement
Ownership
Visual Tracking
Capacity
Stand-Up Meetings
“Done Done”
Customer Examples
Informative Workspace
Accountability
Trust
Forecasting
Roadmaps
Stakeholder Demos
Reporting
Documentation
Improvement
Retrospectives
Team Dynamics
Circles and Soup
Collaboration
Collective Code Ownership
Pair Programming
Mob Programming
Development
Zero Friction
Test-Driven Development
Refactoring
Spike Solutions
Performance Optimization
Design
Simple Design
Reflective Design
Continuous Design
Evolutionary Design
Evolutionary Architecture
DevOps
Build for Production
Trunk-Based Development
Feature Toggles
Continuous Integration
Continuous Deployment
Quality
Slack
Customer Review
Ubiquitous Language
Exploratory Testing
Root-Cause Analysis
Sharpen the Saw
No Bugs
Optimizing
Autonomy
Discovery
Experimentation

XXX Add footer to above table: ✓ = Study this practice carefully. • = You could be involved with this practice.

XXX Update both tables when final set of practices is complete. Current entries are rough drafts and should be thought through more carefully.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Chapter 2: Why Agile?

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Why Agile?

In 1986, Fred Brooks famously predicted that there were no silver bullets: that by 1996, no single technology or management technique would offer a tenfold increase in productivity, reliability, or simplicity. [Brooks 1995] None did.

Agile isn’t a silver bullet, either.

Agile’s benefits come from working differently, not from working faster.

In fact, Agile isn’t really about increasing productivity. Its benefits—even the ability to release software more frequently—come from working differently, not from working faster. Although anecdotal evidence indicates that Agile teams have above-average productivity,1 that shouldn’t be your primary motivation. Your teams will need time to learn Agile. While they learn—and it will take a quarter or two—they’ll go slower, not faster. In addition, emphasizing productivity might encourage your teams to take shortcuts and be less rigorous in their work, which could actually harm productivity.

1See, for example, [Van Schooenderwoert 2006], [Mah 2006], and [Anderson 2006].

There’s no point in Agile for the sake of Agile. Instead, ask yourself two questions:

  1. Will Agile help us be more successful?

  2. What will it take to achieve that success?

When you can answer these questions, you’ll know whether Agile is right for you.

Let’s Talk Success

When I was a kid, I was happy to just play around with computers. I loved the challenge of programming. When I got a program to work, it was a major victory. Back then, even a program that didn’t work was a success of some sort, so long as I had fun writing it. My definition of success centered on personal rewards.

As I gained experience, my software became more complicated. I often lost track of how my programs worked, and I had to abandon some programs before they were finished. I began to believe that maintainability was the key to success—an idea that was confirmed as I entered the workforce and began working with teams of other programmers. I prided myself on producing elegant, maintainable code. Success meant technical excellence.

Despite good code, some products flopped. Even impeccably executed code could elicit yawns. I came to realize that my teams were part of a larger ecosystem involving dozens, thousands, or even millions of people. My software needed to satisfy those people... particularly the ones signing my paycheck. In fact, for the people funding the work, the value of the software had to exceed its cost. Success meant delivering value to the organization.

At each step along my journey, my understanding of the world expanded. For a while, I found myself perplexed by the companies that achieved massive organizational success despite creating technical messes and treating their people terribly. Now I've come to realize that my previous conclusions weren’t wrong. They were just incomplete.

  • Organizational success makes a company;

  • Lack of technical success breaks a company;

  • And personal success glues it together.

You need all three types of success to get the best results.

A Venn diagram showing three circles, marked “Organizational Success,” “Technical Success,” and “Personal Success.” There’s a star in the center where all the circles intersect.

Figure 1. Types of success

Enter Agility

Will Agile help your teams be more successful? It could. Agile development focuses on achieving personal, technical, and organizational successes. If you’re having trouble with any of these areas, Agile can help.

Organizational Success

Organizational success is crucial. Without it, the money will eventually dry up. Organizational success doesn’t just mean dollars and cents, though; see the “What Do Organizations Value” sidebar.

Agile teams achieve organizational success by focusing on delivering value.

Agile teams achieve organizational success by focusing on delivering value. Agile teams also validate expectations early, so if an idea won’t provide good value, you’ll find out early enough to stop working on it before your organization has spent much money.

Specifically, Agile teams increase value by seeking out business expertise, getting frequent user and customer feedback, and focusing development on the core business purpose of their team. Agile teams release their most valuable features first and release updates frequently. When business needs change, Agile teams change direction to match. The best Agile teams actively seek out opportunities to adapt their plans.

Agile teams decrease costs as well. They do this partly via technical success; the best Agile teams generate only a few bugs per month. They also eliminate waste by cancelling bad products early and replacing expensive development practices with simpler ones. Agile teams communicate quickly and accurately, and they’re not bottlenecked by the work of a single team member. They regularly improve their process and code, making their software easier to maintain and enhance over time.

Technical Success

Without technical success, costs rise and your teams’ ability to make changes plummets.

Technical success grants teams the ability to cost-effectively enhance and maintain their software. Without technical success, costs rise and your teams’ ability to make changes plummets. Eventually, they have to throw their software away and rewrite it. It can take years for the replacement to achieve feature parity with the old code. Meanwhile, they’re spending buckets of money just to catch up with where they used to be, all while their customers become increasingly frustrated with their lack of progress.

I highlight two Agile approaches to technical success in this book. The first is Extreme Programming (XP), an Agile method that’s particularly adept at technical successes. In XP, programmers work closely together, which helps them keep track of the nitpicky details necessary for great work. Programmers continuously integrate their code, which enables the team to release their software whenever it makes the most business sense. The whole team focuses on finishing each feature completely before starting the next, which prevents unexpected delays and allows the team to change direction at will.

In addition to the structure of development, XP includes advanced technical practices that lead to excellence. The most well-known practice is test-driven development, which helps programmers write code that does exactly what they intend. XP teams also create simple, ever-evolving designs that are easy to modify when plans change.

I also include techniques from the DevOps movement,2 which brings XP’s ideas into the modern era of cloud-based software. Agile teams using XP and DevOps techniques create high-quality, low-defect software that’s easy to modify, maintain, and operate.

2Not to be confused with DevOps tools, which dominate much of the discussion around DevOps.

Personal Success

Personal success is, well, personal. Most people need more than a paycheck to feel satisfied, and they don’t always complain when they’re unhappy. In the best case, they find a new job, taking their accumulated organizational knowledge with them. In the worst case, they quietly sabotage the company’s efforts as they agitate for work that’s more satisfying.

Agile development won’t satisfy everyone’s needs, and it’s different enough that it can be uncomfortable at first. However, once people get used to it, they find a lot to like about Agile:

Executives and senior management
People focused on the big picture appreciate Agile teams’ emphasis on providing visibility, a solid return on investment, and long-lived code.
Users, stakeholders, domain experts, and product managers
People who care about what the software can do appreciate their ability to influence the direction of development; teams’ focus on delivering useful and valuable software; and increased delivery frequency.
Product and project managers
People interested in delivering a valuable product appreciate their ability to change direction as business needs change; teams’ reliability; and improved stakeholder satisfaction.
UX and graphic designers
People who care about user experience appreciate their ability to influence development plans; teams’ willingness to iterate on ideas; and faster, more accurate implementation of design ideas.
Programmers
People who write code appreciate the improved quality of life that results from technical excellence; their improved influence over forecasts and schedules; and their autonomy.
Testers
People who focus on quality appreciate being integrated with the rest of the development team; their ability to influence quality at all stages of development; and more challenging, less repetitive work.
Operations
People who keep software running appreciate being included in development decisions; having operational requirements included in development plans; programmers’ focus on providing actionable telemetry and logging; and teams taking responsibility for post-release quality and up-time.

Working on Agile teams has provided some of the most enjoyable moments in my career. Imagine the camaraderie of a team that works together to identify and deliver products of lasting value, with each team member enthusiastically contributing to a smooth-running whole. Imagine how it feels to take responsibility for your areas of expertise, whether technical, business, or management, with the rest of the team trusting your professional judgment and ability. Imagine how pleasant it is to address the frustrations of your work and see quality improve over time.

Agile development changes the game. Developing and delivering software in a new way takes a lot of work and thought. Yet if you do it consistently and rigorously, you’ll experience amazing things: you’ll ship truly valuable software on a regular basis. You’ll demonstrate progress multiple times per month. You’ll have the most fun you’ve ever had in software development.

Enough talk. Let’s make it happen.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Chapter 1: What is Agile?

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

What Is Agile?

Agile is everywhere. And paradoxically, nowhere.

In the 20 years after the Agile freight train roared into software developers’ consciousness, the number of companies calling themselves “Agile” increased by orders of magnitude. The number of teams actually taking an agile approach to their work? Not so much. “Agile,” the easily-repeated name, is enormously successful. The ideas behind Agile—well, most of them are ignored.

Let’s fix that.

Agile’s Genesis

In the 1990s, software development was believed to be in crisis. They actually called it that: “The Software Crisis.” Software projects were over-budget, late, didn’t meet requirements, and—according to the oft-quoted and ominously named “CHAOS Report”—nearly a third of them were cancelled outright. [Standish 1994]

Agile wasn’t a response to this crisis. Far from it. Agile was a response to the response.

To bring software development under control, big organizations had created highly detailed processes that defined exactly how software was to be created. Everything was tightly controlled so that no mistakes could be made. (In theory, anyway.)

First, business analysts would interview stakeholders and document the system requirements. Next, software architects would read the requirements documents and create detailed design documents specifying every component of the system and how they related to each other. Then programmers would convert the design documents to code. In some organizations, this was considered low-skill work—just a mechanical translation exercise.1 Meanwhile, test leads would use the same documents to generate test plans, and when coding was done, armies of QA personnel would manually follow those test plans and report variances as defects. After each phase, everything would be carefully documented, reviewed, and signed off.

1In fact, programming was considered such a mechanical activity that, for a while, there was a movement called CASE—Computer Aided Software Engineering—to automatically convert architectural diagrams into code.

These phase-based approaches came to be called “waterfall development,” or “phase-gate development.”2 If they sound like a ridiculous straw-man, well, consider yourself fortunate. Not every company used a document-heavy, phase-based process in the ’90s, but it was widely recognized as a logical and sensible way to work. Of course you needed to define requirements, then design, then implement, then test. Of course you needed to document every phase. This was discipline. This was engineering. How else could you possibly succeed?

2Waterfall is often mistakenly attributed to a 1970 paper by Winston Royce. But phase-based approaches date back to the 1950’s, and Royce’s paper was largely ignored until the late 1980’s, when it was used to describe what people were already doing. Laurent Bossavit explores the mischaracterizations of Royce’s paper, along with other common software development myths, in his superb book, The Leprechauns of Software Engineering: How Folklore Turns Into Fact and What To Do About It [Bossavit 2013] (Chapter 7).

Born Out of Crisis

Big companies defined their processes in excruciating detail. Roles, responsibilities, document templates, modeling languages, change control boards... every aspect of development was strictly defined and controlled. If a project didn’t succeed—and, according to the CHAOS Report, less than a sixth of them did—it was because the process needed more detail, more documents, more sign-offs. It resulted in a massive amount of documentation. Martin Fowler called it “The Almighty Thud.” [Fowler 1997]

This wasn’t a great way to work. It was bureaucratic and dehumanizing. Skill didn’t seem to matter as much as adherence to process. Programmers felt they were interchangeable cogs in an impersonal machine. It didn’t even work all that well.

So several people created simpler, slimmer, and less prescriptive methods for developing software. They were called “lightweight methods” in contrast to the heavyweight methods used by big companies. These new methods had names like “Adaptive Software Development,” “Crystal,” “Feature-Driven Development,” “Dynamic Systems Development Method,” “Extreme Programming,” and “Scrum.”

By the late ’90s, these methods were attracting serious attention. Extreme Programming, in particular, saw an explosion of grass-roots interest among programmers. In 2001, 17 of the lightweight methodology proponents met at a ski resort in Utah to discuss unifying their efforts.

The Agile Manifesto

“I personally didn’t expect this particular group of [people] to ever agree on anything substantive,” Alistair Cockburn said later.

And, in fact, after two days, they only accomplished two things: the name “Agile,” and a statement of four values (see the “Agile Values” figure). During the following months, over email, they hashed out twelve accompanying principles (see the “Agile Principles” figure).

This was the Agile Manifesto. It changed the world. So, as Alistair went on to say, they did agree on something substantive after all.3

3Alistair Cockburn, quoted by Jim Highsmith in [Highsmith 2001]. The full quote is, “I personally didn’t expect that this particular group of agilites to ever agree on anything substantive... Speaking for myself, I am delighted by the final phrasing [of the Manifesto]. I was surprised that the others appeared equally delighted by the final phrasing. So we did agree on something substantive.”

Agile Values

Figure 1. Agile Values

Agile Principles

Figure 2. Agile Principles

If your teams embody the Agile philosophy, then they’re Agile. If they don’t, they’re not.

But there was no unified Agile method. There never has been, and never will be. Agile is three things: the name, the values, and the principles. That’s it. It’s not something you can do. It’s a philosophy. A way of thinking about software development. You can’t “use” Agile or “do” Agile... you can only be Agile. Or not. If your teams embody the Agile philosophy, then they’re Agile. If they don’t, they’re not.

The Essence of Agile

Martin Fowler has made a career out of turning complicated software topics into well-considered, even-handed explanations. His explanation of “The Essence of Agile Software Development” is one of the best:

Agile Development is adaptive rather than predictive; people-oriented rather than process-oriented.4

4Fowler has expressed this idea in many ways over the years. It originated in [Fowler 2000].

Martin Fowler

Adaptive rather than predictive

Remember the CHAOS Report, which said that only one-sixth of software projects were successful? It had a very specific definition of success:

Success
The project is completed on-time and on-budget, with all features and functions as originally specified.
Challenged
The project is completed and operational but over-budget, over the time estimate, and offers fewer features and functions than originally specified.
Impaired
The project is canceled at some point during the development cycle.

These definitions illustrate the predictive mindset perfectly. They’re all about conformance to plan. If you did what you said you were going to do, you were successful. If you didn’t, you weren’t! Easy.

It makes sense at first. But look closer. There’s something missing. As Ryan Nelson wrote in CIO Magazine [Nelson 2006]:

Projects that were found to meet all of the traditional criteria for success—time, budget, and specifications—may still be failures in the end because they fail to appeal to the intended users or because they ultimately fail to add much value to the business.

... Similarly, projects considered failures according to traditional IT metrics may wind up being successes because despite cost, time or specification problems, the system is loved by its target audience or provides unexpected value. For example, at a financial services company, a new system... was six months late and cost more than twice the original estimate (final cost was $5.7 million). But the project ultimately created a more adaptive organization (after 13 months) and was judged to be a great success—the company had a $33 million reduction in write-off accounts, and the reduced time-to-value and increased capacity resulted in a 50 percent increase in the number of concurrent collection strategy tests in production.

Agile teams define success as delivering value, not conforming to a plan.

Agile teams define success as delivering value, not conforming to a plan. In fact, truly Agile teams actively look for opportunities to increase value by changing their plans.

Look back at the Manifesto (see the “Agile Values” figure and the “Agile Principles” figure). “Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.” Take a moment to really study the Agile values and principles. How many relate to delivering valuable software and adapting to feedback?

People-oriented rather than process-oriented

Heavyweight processes tried to prevent errors by carefully defining every aspect of software development. By putting the “smarts” in the process, individual skill became less important. In theory, you could apply the same process over and over, with different people, and get the same results. (Come to think of it, they kind of did. Just not the results they wanted.)

Agile says people are the most important factor in software development success.

Agile says people are the most important factor in software development success. Not just their skills, but all aspects of their humanity. How well team members work together. How many distractions they encounter. How safe they feel. Whether they’re comfortable voicing dissent, and whether they feel motivated by their work.

Agile teams have a process—every team does, even if it’s implicit—but the process is in service of the humans, not the other way around. And Agile teams are in charge of their own process. When they think of a better way of working, they change it.

Look at the Manifesto again (see the “Agile Values” figure and the “Agile Principles” figure). “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.” Which other values and principles relate to putting people first?

Why Agile Won

In the first ten years after the Manifesto, Agile faced enormous criticism. It was “undisciplined,” critics said. “It could never work.” Another ten years after that, the critics were silent. Agile was everywhere, at least in name. Heavyweight waterfall methods were effectively dead. Some younger programmers had trouble believing anybody ever could have worked that way.

It’s not that phase-based processes are inherently broken. They have their flaws, sure, but if you keep them slim and operate in a well-understood domain, waterfall-style methods can work. The problem was the heavyweight approaches big companies used. Ironically, the processes designed to prevent problems actually caused many of the problems organizations were seeing.

Learning and responding to change are at the heart of what Agile is all about.

It’s very difficult to imagine how software will work before you actually use it, and it’s even harder to think of absolutely everything your software needs to do. This is doubly true for people who aren’t actively involved with software development. As a result, it’s critically important to get working software in front of people as soon as possible. You need to get feedback about what’s missing or wrong, then change your plans based on what you learn. As the Manifesto says, “working software is the primary measure of progress.” Learning and responding to change are at the heart of what Agile is all about.

Those heavyweight processes put so much emphasis on process controls and documentation and sign-offs that they incurred a huge amount of delay and overhead. They took years to produce working software, and they had nothing concrete to show until near the end. Instead of welcoming change, they actively worked to prevent change. They actually had a dedicated part of the process, the Change Control Board, whose primary purpose was to say “no” to change requests. (Or, more accurately, “yes, but it will cost you.”)

All of this added up to projects that spent years in development before they had anything to show. When they did, it was too late and too expensive to make changes. They ultimately shipped software that didn’t do what customers needed.

Agile teams show working software, not documents.

Although there are a variety of approaches to Agile—and some of them are more about co-opting a popular name than following the actual philosophy—one thing they all have in common is a focus on making progress visible and allowing stakeholders to make course corrections as they go. This seems like a small thing, but it’s incredibly powerful. It’s why we no longer hear about the Software Crisis. Software is still late. It’s still over budget. But Agile teams show working software, not documents. Right from the beginning. And that’s huge.

There’s more to Agile than just providing visibility. But this one thing? This was enough. It’s why everybody wanted Agile.

Why Agile Works

Agile’s first break-out success was Extreme Programming (XP), a method with the slogan “Embrace Change.” XP mixed a healthy dose of philosophizing about software development with a pragmatic emphasis on making a difference for people involved. As the preface to the first XP book states:

XP makes two sets of promises.

  • To programmers, XP promises that they will be able to work on things that really matter, every day. They won’t have to face scary situations alone. They will be able to do everything in their power to make their system successful. They will make decisions that they can make best, and they won’t make decisions that they aren’t best qualified to make.

  • To customers and managers, XP promises that they will get the most possible value out of every programming week. Every few weeks they will be able to see concrete progress on goals they care about. They will be able to change direction of the project in the middle of development without incurring exorbitant costs.

In short, XP promises to reduce project risk, improve responsiveness to business changes, improve productivity throughout the life of a system, and add fun to building software in teams—all at the same time. Really. Quit laughing. [Beck 2000]

Extreme Programming Explained, First Edition

Some people did laugh. But others tried it, and they found that—contrary to common wisdom about how software development was supposed to work—XP really did everything it promised. And so, despite the laughter, XP thrived, and Agile along with it.

XP was the original poster child of Agile, donating a backbone of ideas and terminology that are still in use today. But the strength of Agile community is that it has always been a big tent. Agile isn’t limited to any one method. It’s constantly expanding to include new people and ideas.

Kent Beck’s Extreme Programming formed the seed. Mary and Tom Poppendieck’s Lean Software Development fleshed out the philosophy. Diana Larsen and Esther Derby’s retrospectives changed the way teams improve. Ken Schwaber and Jeff Sutherland’s Scrum democratized Agile, with a straightforward prescription and a network of trainers that spread Agile far and wide. David Anderson’s Kanban broke the bounds of rigid development cycles. The DevOps movement expanded the Agile team and brought XP’s ideas into the cloud era.

There’s so much more. Alistair Cockburn’s emphasis on communicating, cooperating teams. Jim Highsmith’s work on adaptive planning. Lisa Crispin and Janet Gregory’s support for testers. Bas Vodde and Craig Larman’s exploration of large-scale Agile systems. Johanna Rothman’s guidance for managers. Jeff Patton’s story maps. Pat Reed’s financial accounting. Eric Ries’ Lean Startup. And more, and more, and more. The contributions are too numerous to mention.6

6To the many significant contributors I’ve left out: please accept my apologies.

If you take all these ideas and group them into categories, five core concepts appear.

  • Rely on People. Build processes that understand and work with people’s essential humanity. Put decisions in the hands of those who are most qualified to make those decisions. Base your work on healthy, collaborative relationships.

  • Deliver Value. Seek feedback, experiment, and adapt your plans. Focus on producing valuable results. Consider partially-done work a cost, not a benefit. Deliver frequently.

  • Eliminate Waste. Work in small, reversible steps. Embrace the possibility of failure and design your plans to fail fast. Maximize work not done. Pursue throughput rather than efficiency.

  • Seek Technical Excellence. Enable agility via technical quality. Design for what is known, not what is speculated. Start simple and add complexity only in response to actual needs. Create systems that are easy to evolve, even—or especially—in unanticipated directions.

  • Improve Your Process. Experiment with new ideas. Tune and adapt what works. Never assume the established, popular way is the way that’s best for you.

Agile works because people make it work.

Agile is defined by the Manifesto, but the Manifesto is just the starting point. Agile works because people make it work. They take Agile’s ideas, adapt them to their situation, and never stop improving.

Why Agile Fails

Agile started as a grassroots movement. Its initial success was largely driven by programmers seeking better results and better quality of life. As that success grew, Agile’s momentum shifted from the underlying ideas to hype. Rather than saying, “let’s get better results by adapting our plans and putting people first,” organization leaders started saying, “Everybody’s talking about Agile. Get me some Agile.”

The thing is, there is no “Agile” to go get. It’s just a bunch of ideas. There are specific Agile approaches, such as Extreme Programming and Scrum, that will tell you how to be Agile, but you still have to be on board with the underlying philosophy.

And for a lot of organizations, that underlying philosophy—adapting plans and putting people first—is really, really foreign.

The tragedy of the cargo cult (see the “The Cargo Cults” sidebar) is its adherence to the superficial, outward signs of some idea combined with ignorance of how that idea actually works. In the story, the islanders replicated all the elements of cargo drops—the airstrip, the tower, the headphones—but didn’t understand the vast infrastructure that enabled airplanes to arrive.

People want Agile’s cargo, but they don’t understand the underlying philosophy, and often wouldn’t agree with it if they did.

The same tragedy occurs with Agile. People want Agile’s Cargo: better results, more visibility, fewer business failures. But they don’t understand the underlying philosophy, and often wouldn’t agree with it even if they did. They want to buy Agile, but you can’t buy an idea.

What they can buy is the outward signs of Agile. Stand-up meetings! Stories! Tools! Certifications! There’s lots of stuff labelled Agile, and plenty of people eager to sell it to you. It’s often sold as “enterprise-grade,” which is a way of saying “don’t worry, you won’t have to change.” Uncomfortable ideas like “adaptive planning” and “people-centric” are ignored.

And that’s how you get a cargo cult. All the activity; none of the results. The Agile part is missing.

“At my old company they wasted a huge number of man hours in meetings.”

“[Agile] cost an entire team (30+) people their jobs as they produced almost nothing for almost a year.”

“All [Agile] means is developers get shafted when the project changes... the day before delivery.”

“It’s because of Agile there are sh—ty developers and sh—ty products being shipped.”

“I can’t take this Agile crap any more. It’s lunacy.”

comments about Agile from around the Web

The name Agile is everywhere. The ideas of Agile aren’t. It’s become self-perpetuating: for many, the only Agile they know is Cargo Cult Agile.

Icon of person in a suit wearing coconut headphones

Cargo Cult Agile

It’s time to fix that. In the rest of this book, I’ll show you how to apply Agile ideas for real. Keep an eye out for the Cargo Cult Agilists shown in the margin. They’ll show you what not to do.

Ready? Let’s go.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

AoAD2 Practice: Alignment

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Alignment

Audience
Coaches, Whole Team

We have norms that allow us to work together without conflict.

What is a “team?” It’s not just a bunch of people who sit in the same room. It’s not even a group that’s been assigned to work on the same thing.

Interdependency is the hallmark of a team.

A team is a group of people who depend on each other to accomplish a shared goal. That interdependency is the hallmark of a team. It’s what makes teams so successful... and also what makes them so difficult.

You probably remember working on group assignments in school. They tend to be tolerated at best. We’ve all heard the horror stories about one person who ended up doing all the work while the others mooched off their grade.

But we’ve also heard stories of amazing teams. Maybe you’ve had that experience, too: being part of a great sports team, a band, or a volunteer group. When teams work, they’re electrifying.

What’s the difference between bad teams and good teams? Alignment. Team members in an aligned team not only depend on each other to accomplish a shared goal, they’re in agreement about how they’re going to work together.

Chartering Alignment

Allies
Purpose
Context

Your chartering session is a good time to discuss alignment. (See the “Planning Your Chartering Session” sidebar.) Unlike the other parts of the chartering session—purpose and context—stakeholders don’t participate in the alignment discussion. It’s just for team members and people who work closely with the team, such as a product manager.

As with the other chartering discussions, alignment can raise some sensitive topics. It’s best if you have a neutral facilitator. A good facilitator will help mediate conflicts and make sure everyone’s voice is heard.

During your alignment conversation, you’ll learn about the people on the team; create working agreements to guide team behavior; and establish standards.1

1This agenda is based on [Larsen and Nies 2016] (Chapter 6), with some changes. I’ve removed their “simple rules” activity and added the standards discussion.

Get to know each other

Start your alignment discussion by getting to know each other better. If you’ve conducted the context discussion, you’ll have already created a skills inventory. Now it’s time to go deeper.

If you haven’t created a skills inventory yet, you can do so when people answer the first question in this activity.

This activity requires a bit of advance preparation. First, post the team’s purpose and skills inventory in a prominent location. Second, provide a handout with the following questions.

  1. Who am I? Recap the skills, experience, connections, and permissions from the skills inventory, then share some of your positive personal qualities, such as “detail-oriented,” “patient,” or “friendly.”

  2. What’s something people learn about me once they’ve gotten to know me? Possibilities include a hobby, favored vacation destination, or beloved pet.

  3. Why do I believe this group is well-suited to achieving the team’s purpose?

  4. What’s the most important thing that others need to know about working with me effectively?

  5. What’s in it for me? What do I want to get out of being part of this team and accomplishing our purpose?

When you’re ready to start, go around the room, one question at a time, and ask each person to answer. People can skip their turn if they need time to think, but come back to them before moving on to the next question.

Having this discussion will help team members start to see each other as whole people, rather than just names, faces, and titles. It’s most effective when done face-to-face, so if you have a remote team, make an extra effort to have this conversation by video.

Next, prepare a whiteboard, flipchart, or portion of your virtual whiteboard with three columns. Label the first two “Same” and “Different.” Then ask the group to use simultaneous brainstorming (see the “Work Simultaneously” section) to answer two questions: “In what ways are we the same?” and “In what ways are we different?” Put each answer on a sticky note and add it to the board. Prompt participants to go broad, including everything from the obvious to the profound.

People might be reluctant to speak up, at first, so let the brainstorming exercise go on a bit longer than is comfortable. When the ideas trickle to a stop, wait longer—a minute, two, or even three—and allow the silence to stretch out. People will often add a few more ideas that they were holding back, and these last ideas can be the most valuable.

Give everyone a moment to review the sticky notes, then label the third column “DTMD,” for “differences that make a difference.” Ask participants to discuss the differences they’ve brainstormed. Which differences will have the most impact on the team and their ability to achieve their purpose? As people nominate sticky notes, move them from the “Different” column to the “DTMD” column.

After the important differences have been collected, discuss the list and narrow them down to the top 5-10 most important. Take a few minutes for a free-form discussion about what these differences mean for the team.

Diversity of experience leads to diversity of ideas.

This activity can feel awkward or “touchy-feely.” It’s worth doing anyway. The best teams are composed of people with diverse backgrounds and experiences, because diversity of experience leads to diversity of ideas. Those ideas will help fuel your team’s success.

This activity does more than just help team members get to know each other better. By taking the time to understand and embrace your differences, you’re helping to ensure that people with unusual experiences will be comfortable sharing the wisdom gained from those experiences, and you’re ensuring that people with mainstream experiences are aware of the viewpoints their fellow team members can bring to bear.

Create working agreements

Working agreements guide your team’s behavior by describing what you expect from each other. Over time, the agreements will change. As the team matures, some of the working agreements will become second nature, and can be taken off the list. Others will be added, to take advantage of new ideas.

To create your team’s working agreements, start by sharing stories of other teams you’ve worked with. Describe how they worked together, either good or bad. You can do this in round-robin order, or just randomly, according to who wants to talk next.

As people share their experiences, pause to make a note of potential working agreements. They can be anything: behavioral standards, such as “we don’t interrupt when people are talking;” specific practices, such as “we use pair programming for all production code;” work habits, such as “when we change tasks, we drop a note in the group chat;” and more. Include anything that will help your team work together better. Don’t critique the suggestions yet; just collect them.

After you share your stories, check whether you’ve covered the following categories. If you haven’t, suggest working agreements in these categories, too. You won’t have to choose them, but make sure they’re considered.

  • The practices your team will use. (I recommend starting by-the-book with every practice in your team’s chosen zones, as discussed in the “How to Be Agile” chapter.)

  • For remote teams, how you’ll coordinate and collaborate (see the “Designing Remote Collaboration” section).

  • How you’ll make decisions (see the “Work Simultaneously” section, the “Seek Consent” section, and the “Agree to Experiment” section)

  • For in-person teams, how you’ll deal with distracting background noise (see the “Team Room Prequisites” section).

  • For teams that don’t use pairing or mobbing, how you’ll ask for help without being disruptive (see the “Always Ask Always Help” section).

  • The core hours when you can expect people to be available (see the “Team Room Prequisites” section).

Choose working agreements that need attention, not ones the team will do automatically.

After generating the ideas, narrow down the list by using dot voting (see the “Work Simultaneously” section) to select the top five. A few more or less is fine. Tell people to vote for proposals that they think the team needs to pay extra attention to, not the ones they will do automatically.

Make sure everyone’s okay with the final list by conducting a consent vote (see the “Seek Consent” section). If you’re unable to get consent for a particular working agreement, just drop it from the list for now. You’ll be able to revisit it later.

Finish off by rephrasing the working agreements and transferring them to a cleaned-up list. Each working agreement should finish the sentence, “We work together best when...” Describe what to do instead of what not to do. In other words, rather than saying, “We don’t interrupt each other,” say, “We let people finish their thought before adding our own.”

Allies
Team Room

Post the final list of agreements prominently in your team room. You can start using them immediately.

Define standards

Standards are special working agreements that apply to specific types of tasks. Coding standards, UI design guidelines, and operational standards are all examples.

Standards tend to be a source of conflict if they’re not addressed, so it’s good to create them. What’s not as important is their actual content. Over time, you’ll amend and improve them.

Remember, few decisions are irrevocable in Agile; mistakes are opportunities to learn and improve. Ward Cunningham put it well:2

2Quoted at https://en.wikiquote.org/wiki/Ward_Cunningham

It was a turning point in my programming career when I realized that I didn’t have to win every argument. I’d be talking about code with someone, and I’d say, “I think the best way to do it is A.” And they’d say, “I think the best way to do it is B.” I’d say, “Well no, it’s really A.” And they’d say, “Well, we want to do B.” It was a turning point when I could say, ”Fine. Do B. It’s not going to hurt us that much if I’m wrong. It’s not going to hurt us that much if I’m right and you do B, because, we can correct mistakes. So [let’s] find out if it’s a mistake.”

To that end, use these two guidelines to define your standards:

  1. Create the minimal set of standards you can live with.

  2. Focus on consistency and consensus over perfection.

Allies
Done Done

For your first standard, decide what it means for work to be “done.” Start your discussion by asking participants to propose an industry standard as your baseline. (For the definition of “done,” you can use the “Done Done” checklist in this book.) If your company already defines a standard, start with that.

If there are multiple proposals for the baseline standard, take a moment to discuss the options and choose one by consent. Limit your discussion to five minutes. If you can’t agree on one in that time, start without a baseline.

Next, use simultaneous brainstorming to think of additions or changes. Group the ideas using affinity mapping (see the “Work Simultaneously” section), then dot vote to choose the most important categories.

Starting with the category that received the most votes, ask someone to propose a specific standard. Conduct a consent vote, resolve objections, and move on to the next proposal.

Limit each consent discussion to five minutes. If you haven’t reached consent by that time, then just skip that proposal for now. Again, you’ll have a chance to revise your standards later. Limit the entire discussion to 45 minutes.

After you’ve created your definition of “done,” repeat the process with any other standards you need, such as coding standards. You can do this in parallel by splitting up into specialties, such as programming, UX design, and operations. To summarize:

  1. Start by consenting to a baseline standard. (Limit to five minutes.)

  2. Brainstorm additions or changes. Group them into categories.

  3. For each category, consent to a specific standard. (Limit to five minutes.)

  4. Limit entire discussion to 45 minutes.

No matter which standards your group chooses, some are likely to feel jarring or grating at first. Over time, they’ll become more comfortable. Standards are, in many ways, an aesthetic choice. One of the marks of a professional is the willingness to put aside personal aesthetics for a team aesthetic.

Iterating Alignment

As with everything else Agile, your agreements aren’t set in stone. The standards, in particular, need some time to bake. Schedule another meeting to discuss your standards a few days after you start working together, and then another one a few weeks after that. An hour should be enough. This will allow you to try your standards in practice. If there’s still disagreement about a standard after that time, agree to experiment with one approach, then the other, then revisit the question.

Allies
Team Room
Retrospectives

After these initial standards meetings—which you can use to revise your other working agreements, too—you can change your agreements at any time. Just announce your intention to the team, get consent, then change the flip chart or virtual equivalent. Retrospectives are another good time to discuss changes to working agreements.

Your team agreements (the first list, not the standards) is for habits that you’re actively working on establishing. It’s best to limit this list to about five agreements. When an agreement becomes second nature, you can take it off the list and make room for a new agreement.

Adhering to Agreements

Assume your colleagues are professional and well-meaning.

People make mistakes. Assume your colleagues are professional and well-meaning. If someone isn’t following an agreement, assume that there’s a good reason, even if all the evidence is to the contrary. Your challenge is to find that reason and address it. This approach shows respect for others and will improve their respect for you.

Allies
Pair Programming
Mob Programming
Collective Code Ownership

Pairing, mobbing, and collective code ownership all help team members catch mistakes and maintain self-discipline. They also provide a way to discuss questions not addressed by your team’s agreements. They’re also an excellent way to improve your standards; it’s much easier to suggest an improvement after you’ve talked it over with someone first.

Automated enforcement of standards tends to be less effective. Some people use automated tools to check their source code for adherance to coding standards, or automatically reformat code upon check-in. Although this can work when the team is in agreement, teams commonly fall into an over-enforcement trap.

I’m reminded of a time when my team added an automated style checking tool. It blew out our build times and kept throwing false positives. We spent more time on the tool that it ever saved.

Technical tools can’t solve interpersonal problems.

Even worse, people often use tools as a bludgeon to enforce their opinion. Although it’s tempting to come up with a technical solution to interpersonal problems, it won’t work. You have to solve the interpersonal issue first. Tools have no nuance. At best, they’ll paper over the disagreement. They won’t solve it; they’ll just shove it out of sight to fester and grow.

Instead, it’s best to start by assuming good faith. Perhaps the other person misunderstood the agreement, or feels it no longer applies, or something in their life is making it difficult for them to comply.

If someone consistently violates the team’s agreements, talk with them alone to see if there’s a disagreement. Take an attitude of collaborative problem solving. Instead of saying, “Why aren’t you handling nulls like we agreed?” ask, “What do you think about the null-handling standard we agreed on? Should we change it?” Give objections full consideration, raise them with the rest of the team, and consider changing the agreement.

If the objector is on board with the agreement but still isn’t following it, it’s possible that the agreement isn’t appropriate in every situation. Ask about specific cases you’ve noticed. Again, be collaborative, not confrontational. Say something like, “I agree with you about how we should handle nulls. In that case, can you explain what’s happening in this function? I don’t understand why this code doesn’t check for nulls.”

During this discussion, you may learn that the objector doesn’t understand the agreement. By this time, you should be in a good situation to discuss the agreement and what it means. If they’re a junior team member who needs more help, coordinate with the rest of the team to make sure they get plenty of mentoring from more experienced team members.

There’s another possibility for teams new to Agile. Changing work habits is disruptive and can make people feel like they’ve lost control. Sometimes they react by picking small things that they refuse to change. An obstinate desire to stick with a particular standard or communication style, regardless of the wishes of the rest of the team, might be a symptom of this reaction.

In this case, your best solution may be to let the infractions slide for several months. Over time, as team members become more comfortable with the changes in their environment, they’ll relax and be more willing to compromise.

Questions

What if we can’t agree on standards or other working agreements?

It’s possible to pressure people into accepting agreements that they don’t agree with, but it’s not a good idea. The disagreement will just keep coming up in other conversations.

Instead, try to let it go. Is the working agreement really so important? Focus on the things you do agree on. As work progresses, your differences will resolve.

If it’s not something you can safely ignore, your team needs the help of professional mediator. Talk to your coach or manager about finding someone who can help. Your HR department may have someone on staff. In the worst case, it may turn out that your group isn’t suited to be a team, and you’re better off teaming up with different people.

We have pre-existing work that doesn’t fit our standards. Should we fix it?

It’s expensive and risky to spend a lot of time fixing things that aren’t broken, so if your previous work, well, works, go ahead and leave it alone. Bring each piece up to standard when you need to change it.

Some standards, like code formatting, can be automated. Don’t spend too much time on this, but if you can do it easily, it’s fine to do so. Be sure to coordinate automated changes with other team members so their work isn’t disrupted, and separate the automated changes from normal changes so your version control history is easy to read.

Prerequisites

Your team has to be willing to work as a team before they can create effective working agreements. See the “Invest in Change” chapter for more about team buy-in.

Indicators

When your team has effective working agreements:

  • Your team uses your agreements to prevent and resolve conflicts.

  • Your standards improve the readability and maintainability of your code and other production artifacts.

  • Your standards allow team members to more easily understand parts of the system they aren’t familiar with.

Alternatives and Experiments

Some teams work together so well that they don’t need explicit agreements. Their agreements are implicit.

For new teams, though, and even most existing teams, taking the time to explicitly discuss your agreements will help prevent disruptive arguments in the future. The exact format of that discussion isn’t important. The format in this book was chosen because it’s relatively quick and non-confrontational, but you could use another approach.

Stick with the approach in this book until you’ve had experience with several successful alignment discussions. Alignment can be contentious, particularly when teams get down to concrete agreements such as standards, and that’s why this book emphasizes setting aside divisive agreements.

Once you have experience with successful alignment discussions, go ahead and experiment with changes to the format. Would small group discussions or interviews help? Is there any preparation that people could do in advance? Would different exercises be faster or more effective? There’s no limit to what you can try.

Further Reading

Liftoff, Second Edition: Start and Sustain Successful Agile Teams [Larsen and Nies 2016] has much more detail about facilitating chartering discussions. It also includes an additional activity, Simple Rules, that I left out of this book.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.