AoAD2 Practice: Adaptive Planning

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Revised: July 13, 2021

Adaptive Planning

Audience
Product Managers, Customers

We plan for success.

Imagine you’ve been freed from the shackles of predetermined plans. “Maximize our return on investment,” your boss says. “We’ve already talked about the purpose of this team. I’m counting on you to work out the details. Create your own plans and set your own release dates—just make sure we get good value.”

Now what?

Valuable Increments

Build your plans out of valuable increments.

Build your plans out of valuable increments.1 Valuable increments have three characteristics:

1In the first edition of this book, instead of “valuable increment,” I used the term “Minimum Marketable Feature” (MMF) from [Denne and Cleland-Huang 2004]. Valuable increments are the same idea, but I’ve changed the name because not everything that’s valuable is marketable, or a feature.

  1. Releasable. When you finish working on the increment, you can release it and reap its benefits, even if you never work on it again.

  2. Valuable. The increment benefits your organization in some way. (See “What Do Organizations Value” on page XX.)

  3. Incremental. It doesn’t do everything. It’s one step in the right direction.

Don’t confuse “valuable increments” with “potentially shippable increments,” another common term in the Agile community. Potentially shippable increments are about a team’s technical ability to release changes. Valuable increments are about changes that actually make an appreciable difference to your business.

Ally
Continuous Deployment

Similarly, a valuable increment is only “released” when you can achieve its value. Teams using continuous deployment will deploy their software multiple times per day, but it isn’t released until a configuration switch is flipped and the increment is available to its intended audience.

Valuable increments generally fall into these categories:

  • Direct Value. You build something, change something, or fix something that has value. It’s “released” when your organization can reap the benefit. For example, if you think you can improve customer retention by adding a new report, you’ve released the report when real customers can run it.

  • Learning Value. You conduct an experiment that gives you insight into how to increase value. It’s “released” when the experiment is ready to run, including decisions about how to interpret the results. For example, if you think you can increase customer sign-ups by changing your sign-up flow, but you’re not sure which flow is best, you might create an A/B test.2 The experiment has been released when the A/B test is active, the point at which the data will be evaluated has been decided, and the criteria for keeping or discarding the new flow has been determined.

  • Option Value. You create the ability to postpone or change a decision, so that you can take advantage of a valuable opportunity in the future. It’s “released” when you can safely postpone or change the decision. For example, if you think a vendor is trying to create lock-in so they can raise their prices, you might modify your software to also support a second vendor. The option has been released when you can switch between vendors at will.

2A/B testing is when you show different things to different groups of people and evaluate which one had the best results.

Learning and option increments require people to be comfortable with uncertainty and ambiguity, so they tend to be used by Optimizing teams. But any team can use them.

Allies
Stories
Visual Planning
Purpose
Incremental Requirements

You’ll track your increments with stories in your visual plan. For example, the previous examples might be written as “TPS Report,” “sign-up flow A/B test,” and “authentication vendor independence.” If you want to keep more detailed notes, you can, but for most product managers I meet, a short phrase is enough of a reminder. You need to be able to articulate three things:

  1. Why the increment is valuable

  2. How the value relates to the team’s purpose

  3. What “released” looks like, at a high level

The details are decided later, as I’ll discuss in “How to Create Your Plan” on page XX.

Focus on One Increment at a Time

Focusing on one increment at a time will improve delivery speed and increase value.

Stakeholders love it when teams work on multiple ideas simultaneously. It feels like a lot of work is being done, and everything gets to be top priority! So easy. And so very, very wasteful. Focusing on one increment at a time will improve delivery speed and increase value.

Consider a team that has three valuable increments, as shown in figure “Effects of Focusing on Value”. In this simplified example, each increment has equal value: when complete, each will yield $4 in value every month. Each increment takes two months to complete.

Four scenarios. Scenario A shows a team working on three increments simultaneously over six months. They earn $4 for each one in the seventh month for a total value of $12. Scenario B shows a team focusing on one increment at a time. They work on Increment 1 in months 1-2 and it earns $4 in months 3-7. They work on Increment 2 in months 3-4 and it earns $4 in months 5-7. They work on Increment 3 in months 5-6 and it earns $4 in month 7. After month 7, Increment 1 has earned $20, Increment 2 has earned $12, and Increment 3 has earned $4, for a total of $36. Scenario C is similar to Scenario B, but it has six incrementss earning $2 per month. After month 7, Increment 1a has earned $12, Increment 1b has earned $10, Increment 2a has earned $8, Increment 2b has earned $6, Increment 3a has earned $4, and Increment 3b has earned $2, for a total of $42. Scenario D is the same as Scenario C, except Increments 1a and 2b earn $3 per month, Increments 1b and 3a earn $2 per month, and Increments 3b and 2a earn $1 per month. The increments have been reordered so the most valuable increments are done first. After month 7, Increment 1a has earned $18, Increment 2b has earned $15, Increment 1b has earned $8, Increment 3a has earned $6, Increment 3b has earned $2, and Increment 2a has earned $1, for a total of $50.

Figure 1. Effects of focusing on value

In Scenario A, the team works on all three increments simultaneously. It takes them six months to finish all three. When they’re done, they start collecting $12 every month.

In Scenario B, the team focuses on one increment at a time. They release their first increment after two months—one-third the time of Scenario A. It starts earning money while they work on the next increment. By the end of the seventh month, they’ve earned $36 in the time it took Scenario A to earn $12. And that’s ignoring the costs of task switching and the benefits of getting to market sooner. It’s free money.

The more frequently you release, the more value you get.

The more frequently you release, the more you get, as Scenario C shows. It’s the same as Scenario B, except that the team has figured out how to slice each of their increments in half. Instead of releasing a $4 increment in two months, they release a $2 increment in one month. After seven months, they’ve earned $42.

Of course, some ideas are more valuable than others. Scenario D shows what happens when you split the most valuable parts out and work on them first. This scenario is the same as Scenario C, but the increments are worth different amounts. Reordering increments to release the most valuable ones first allows Scenario D to earn $50. Three months of free money relative to Scenario A.

This is the essence of Focusing fluency. Focus on small, valuable increments. Focus on releasing one at a time. Focus on the most valuable ideas first. After each release, take advantage of any new information you’ve learned, adapt your plans, and Focus on the highest value that remains.

The scenarios in figure “Effects of Focusing on Value” are simplified. Software By Numbers [Denne and Cleland-Huang 2004] (pp. 19-24) has a more sophisticated example based on a real product, shown in table “Realistic Example of Focusing on Value”. In their example, the authors convert a five-year project with two end-of-project releases (Scenario A) into five yearly releases ordered by value (Scenario B). The team’s productivity was the same in each scenario.

Table 1. Realistic example of focusing on value

Scenario AScenario B
Total Cost$4.312 million$4.712 million
Revenue$5.600 million$7.800 million
Investment$2.760 million$1.640 million
Payback$1.288 million$3.088 million
Net Present Value @ 10%$0.194 million$1.594 million
Internal Rate of Return12.8%36.3%

Scenario A is a marginal investment with a 12.8% rate of return. It requires an investment of $2.8 million and yields profits of $1.3 million. Considering the risk of software development, the investors can put that money to better use elsewhere.

Scenario B—the same product released more often—is an excellent investment with a 36.3% rate of return. Although Scenario B costs more, because it conducts more releases, those releases allow the product to be self-funding. As a result, it requires a smaller investment—$1.6 million—and yields profits of $3.1 million. This scenario is well worth funding.

Look at these examples again. Each shows impressive increases in value. Yet nothing changes except the way the teams release their software.

Slice Your Increments

As the examples showed, the more finely you can slice your increments, the more value you can extract from your team’s work. Beyond that, though, each increment represents an opportunity to change direction without incurring waste. If you change direction in the middle of working on a increment, you’re left with partially-completed work you either have to set aside or throw away. Once an increment is done, you can change direction without wasting any work.

The smaller your increments, the more agile you can be.

The smaller your increments, the more frequently you can adapt your plans, and the more agile you can be. In a perfect world, your increments should be broken down to the smallest fundamental pieces that can still have releasable value.

In reality, it’s hard to make your increments that small, especially at first. As time passes, you’re likely to see ways to split your increments further. It’s okay to start with your best guess and split them later. Agile’s iterative. You’ll have plenty of opportunities to improve your increments.

Release Early, Release Often

You can release a valuable increment as soon as it’s finished, or you can wait to bundle multiple increments together into a single release. Although releasing immediately has the highest value, sometimes it can be more effective to market multiple increments together rather than dribbling them out one at a time. Similarly, if there are user interface changes or other release-related costs, it can be easier to absorb those costs all at once rather than continuously making small changes.

Ally
Continuous Deployment

Some teams use continuous deployment to deploy their code multiple times per day, but deploying code isn’t the same as releasing an increment. An increment isn’t “released” until it’s available to be used. For teams using continuous deployment, “releasing” usually involves changing a configuration setting.

Sometimes teams bundle increments together because technical constraints make frequent releases too costly. It’s okay to do this, if you must, but be honest with yourself about the reasons. In addition to high release costs, you’re also incurring the costs of increased work in progress. You can eliminate both costs by investing in Delivering fluency.

The release train always leaves on time.

Some teams use release trains to schedule their releases. A release train is a pre-scheduled series of releases, such as the first Monday of every month. Finished increments “get on the train” and are included in that release. The rest wait for the next train. It doesn’t matter how close they are to being done; the train always leaves on time.

The Scaled Agile Framework (SAFe) defines an “Agile Release Train” as a container for teams, which is an unfortunate redefinition of the term.3 Here, I’m using the simpler, original meaning. A release train is just a predefined release schedule. Don’t overthink it.

3The term “release train” dates to well before SAFe. The earliest reference I’m aware of is Sun’s Software Development Framework from 1993, which defined the term in the same way I do here. That document isn’t publically available, but [Rothman 1998] references the term and uses a similar definition. Many thanks to Howard Fear and Dave Hounslow for digging up these references.

Release trains have a lot of benefits. They give marketers an event to celebrate. They give users a date to anticipate. They give stakeholders certainty about when they can expect progress, and they take pressure off teams, who—so long as their increments are smaller than the release cycle—can commit to release dates without worrying about exactly what they’ll release on each date.

On the other hand, a decision to delay an increment, either by bundling it into a one-off release or using a release train, is a decision to delay value. Organizations also tend to use release trains to paper over technical and organizational deficiencies that prevent more frequent releases. When you think about your release strategy, trade off the benefits of bundling increments into releases with the cost of delaying their value, and be honest with yourself about your reasons for delaying.

Your First Increment

Your first increment can be tricky. It needs enough content to be interesting, but not so much that you delay your release.

One way to think about your first increment is to think in terms of a Minimum Viable Product (MVP). Contrary to the common understanding of this term, an MVP isn’t the smallest product you can successfully release. Instead, it’s a way of validating your product ideas. Eric Ries defined the term in his influential book, The Lean Startup:

A minimum viable product (MVP) helps entrepreneurs start the process of learning as quickly as possible. It is not necessarily the smallest product imaginable, though; it is simply the fastest way to get through the Build-Measure-Learn feedback loop with the minimum amount of effort.

Contrary to traditional product development, which usually involves a long, thoughtful incubation period and strives for product perfection, the goal of the MVP is to begin the process of learning, not end it. Unlike a prototype or concept test, an MVP is designed not just to answer product design or technical questions. Its goal is to test fundamental business hypotheses. [Ries 2011] (pp. 95-96)

Eric Ries

An MVP isn’t necessarily a release, or even a product, in the traditional sense. It’s an experiment, and you can have more than one. As such, true MVPs are most often used by Optimizing teams.

Whether your first increment is an MVP in Eric Ries’ sense of the word, or just the smallest increment that buyers and users will love, is up to you.

Adapt Your Plans

You’ll think of your first increments before a single line of code is written. This is when you know the least about what will make the software valuable. You might know a lot, but you’ll always know more after you talk with stakeholders, show them demos, and conduct actual releases. Over time, you’ll discover that some of your initial ideas about value were incorrect. If you change your plan to reflect what you’ve learned—if you adapt—you’ll create a more valuable result.

Think of your plan as a plan for learning as much as it is a plan for building.

To increase the value of your software, create opportunities to learn. Think of your plan as a plan for learning as much as it is a plan for building. Focus on what you don’t know. What are you uncertain about? What might be a good idea? Which good ideas can you prove in practice? Don’t just speculate—create increments for learning. Test each uncertainty, then adapt your plan.

For example, if you were creating an online word processor, you might not be sure how extensive your support for importing Microsoft Word documents should be. Some sort of support is necessary, but how much? Supporting all possible Word documents would take a long time to implement. It would take time away from adding other, possibly more valuable features. Too little support could damage your credibility and cause you to lose customers.

To test this uncertainty, you could add a rudimentary import feature to your software (clearly marked “experimental”), release it, and have it report the types of documents real users try to import. The information you gather will help you adapt your plan and increase your team’s value.

Users of web-based software are used to “beta” web applications, so releasing an experimental, incomplete feature is possible in that context. A product with less forgiving users may require the use of a pre-release program, focus groups, or some other feedback mechanism.

How to Create Your Plan

Valuable increments are the building blocks of your plan, but they don’t have any details. Those details take a lot of time and work. Worse, when you adapt your plans, some of that work could be thrown away.

Ally
Incremental Requirements

To reduce waste and make planning easier, use rolling-wave planning to plan at the last responsible moment. In rolling-wave planning, detail is added gradually, just before it’s needed.

A chart showing a funnel on its side. The wide end of the funnel is on the left, labelled “approximate, broad view,” and the narrow end is on the right, labelled “detailed, narrow view.” The funnel is divided into several regions. Each region is labelled, and the regions are separated by a dividing line with a time and the caption “time until work is done.” From left to right, the regions and dividing lines are: “purpose” (region), “six months” (divider), “possible valuable increments” (region), “three months” (divider), “smallest valuable increments” (region), “one month“ (divider), “‘just right’ stories” (region), “one week“ (divider), “tasks and detailed requirements” (region), and “done“ (final divider).

Figure 2. Planning horizons

Allies
Purpose
Visual Planning
The Planning Game
Task Planning
Incremental Requirements

The distance each level of detail looks into the future is called its planning horizon. An Agile plan has multiple planning horizons, as illustrated in figure “Planning Horizons”:

  1. Start with the purpose for the team, which will include the team’s mission.

  2. Use visual planning to create a map of possible valuable increments for achieving the mission.

  3. Continue using visual planning to break the first few increments into the smallest valuable increments you can think of.

  4. Use the planning game to break the first small increments down even further, into stories that are sized “just right.”

  5. Use task planning to break the first few stories into development tasks.

  6. Just prior to starting development on a story, use incremental requirements to determine requirements details.

“Your First Week” on page XX describes how to get started. Once the plan is in place, you’ll use a pull system to maintain it. In a pull system, rather than doing work at predefined intervals, you do work in response to demand. In this case, the “pull” comes from finishing tasks.

  1. When your team finishes its tasks, you’ll need more tasks. You’ll use task planning to pull stories out of the visual plan and onto your task board, where you’ll break them into tasks.

  2. This could lead you to needing more stories. When you do, you’ll schedule a planning game session and pull stories out of your next small, valuable increment.

  3. When you finish an increment, you’ll use visual planning to pull a new small increment out of your possible increments.

  4. And when you need more ideas for possible increments, you’ll pull them out of your purpose and mission.

  5. And finally, when you’re close to completing your mission, you’ll go back to your sponsor and pull a new mission out of your team vision and purpose.

Different skills are needed for each level of detail, as figure “Planning Skills” shows.

This chart shows the same image as the “planning horizons” figure, except the funnel and regions are in the background. The names of several skills are overlaid on the funnel, with arrows showing which regions the skills apply to. From left to right, they are: “sponsor and key stakeholders,” which overlays the “purpose” and “possible valuable increments” regions; “product management,“ which overlays “purpose” (partially); “possible valuable increments,” “smallest valuable increments,” and “‘just right’ stories” (partially); “domain expertise and UX design,” which overlays “possible valuable increments” (partially), “smallest valuable increments,” “‘just right’ stories,” and “tasks and detailed requirements”; “business-focused testing,” which overlays “smallest valuable increments (partially), “‘just right’ stories,” and “tasks and detailed requirements”; and finally “programming, operations, and technology-focused testing,” which overlays “smallest valuable increments” (slightly), “‘just right’ stories,” and “tasks and detailed requirements.”

Figure 3. Planning skills

Balancing Adaptability and Predictability

Each level of detail in your plan has an associated planning horizon. For example, figure “Planning Horizons” shows that tasks are planned for the next week; stories are planned for the next month; small increments are planned for the next three months; and possible increments are planned for the next six months.

Choose your planning horizons based on how you want to trade off adaptability and predictability.

Those horizons are just an example. You should choose your own planning horizons based on how you want to trade off adaptability and predictability. For example, if your plans change a lot, you might create “just right” stories only two weeks in advance, and small increments just a month in advance. On the other hand, if your stakeholders need a lot of certainty, you could create “just right” stories three months in advance, and small increments six months in advance.

Allies
Roadmaps
Forecasting

The longer your planning horizons, the more work you’ll throw away when your plan changes, and the more people will resist making changes. On the other hand, longer planning horizons are often needed for good roadmaps, and your story planning horizon determines how far into the future your release forecasts can see.

Ultimately, the choice of planning horizons is a tradeoff between less waste and greater agility (shorter planning horizons) and more certainty and predictability (longer planning horizons). There’s no wrong answer; just a choice between tradeoffs. If you aren’t sure what to choose, start with the horizons shown in figure “Planning Horizons”.

Adaptive Planning and Organizational Culture

Does the idea of spending two months travelling in a foreign country without a detailed plan sound scary? In practice, it was easy and relaxing, but when I tell the story of our adaptively-planned trip to Europe (see “Adaptive Planning in Action” on page XX), people get nervous.

Organizations often have a similar reaction to adaptive planning. An adaptive plan works to achieve the team’s purpose. However, just as the trip my wife and I took achieved its purpose—“have fun visiting a lot of European cities”—without choosing specific cities in advance, an adaptive team will achieve their purpose even though they won’t be able to say exactly what they’ll deliver.

No aspect of Agile challenges organizational culture more than adaptive planning.

No aspect of Agile challenges organizational culture more than adaptive planning. It requires changes not only to the development team, but to reporting, evaluation, and governance. The choice of adaptive planning extends to surprisingly diverse parts of your stakeholder community, and people often have a shocked or emotional reaction to the idea.

As a result, you may not be able to influence a change to adaptive planning. Unless you have the support of senior leadership, any change that does occur will probably be slow and gradual. Even with leadership support, this change can take time.

Ally
Forecasting

Work within your organization’s culture to introduce adaptive planning gradually. Use rolling-wave planning, but set your planning horizons to match the organization’s expectations. You may need to make forecasts as well, which typically requires Delivering fluency. As your stakeholders gain trust in your ability to deliver, shorten your planning horizons and migrate towards a more adaptive plan.

Questions

We need to commit to specific release dates. What should we do?

See “Forecasting” on page XX. If you use date and scope forecasts, you may need to increase your planning horizons, and you’ll typically need Delivering fluency for your forecasts to be useful.

If we don’t plan our releases in detail, what should we tell our stakeholders about our plans?

Although you may not plan out all the details of your releases in advance, you’ll still be able to share a roadmap with stakeholders. If they need a lot of detail, you might need to increase your planning horizons. See “Roadmaps” on page XX.

If we use short planning horizons, how can we be sure we’ll fulfill our team’s purpose?

If you’re not sure you can meet your purpose, focus your plan on discovering whether you can. You may need to create increments for learning, increase your planning horizons, or create a small, limited-availability release to test crucial concepts. The details depend on your situation, so if you’re not sure what to do, ask a mentor for guidance.

Prerequisites

Adaptive planning requires manager and stakeholder buy-in. (See chapter “Invest in Change”.) Any team can plan in terms of valuable increments. Beyond that, it depends on how much buy-in you have.

Working on one increment at a time is a smart, easy way to increase your team’s value. Despite its usefulness, working on one thing at a time is anathema to some stakeholders. Proceed with caution.

Releasing frequently requires your customers and users to be able to accept frequent releases. This is a no-brainer for most web-based software, because users don’t have to do anything to get updates. Other types of software may require painful software rollouts, and some even require expensive certification testing. That can make frequent releases difficult.

Creating experiments, options, and MVPs requires that your organization accept uncertainty, and trust that your team has sufficient market expertise to make good decisions. This often requires your team to have Optimizing fluency.

Rolling-wave planning requires a clear purpose and regular updates to the plan. Use it when you can reliably revisit the plan at least once per week. Be sure your team includes people who have the skills to update and revise the plan.

Adapting your plans requires that your organization think of success in terms of value rather than “delivered on time, on budget, and as specified.” This can be a tough idea for some organizations to swallow. You may be able to assuage fears about adapting plans by committing to a specific release date but leaving the details of the release unspecified.

Finally, be cautious of plans without a release in the next three months. Although those release goals shouldn’t be treated as commitments, it’s easy to wander off course without the checkpoint and urgency of a near-term goal.

Indicators

When you create, maintain, and communicate a good plan:

  • The plan shows how the team will achieve its purpose, or learn how to achieve it.

  • Team members are confident the plan is achievable.

  • You release value regularly and consistently.

When you adapt your plan well:

  • You consistently seek out opportunities to learn new things about your plan, your product, and your stakeholders.

  • As you learn, you modify your plan to take advantage of new insights.

Alternatives and Experiments

Adaptive planning increases value by being flexible in planning and strategic in releases. Look for opportunities to reduce time spent on plans that are discarded, quicken the feedback loop, improve your plans more often, and shorten time to value.

Ally
Incremental Design

If you don’t have a Delivering team, you might run into questions about how to plan for technical infrastructure. [Denne and Cleland-Huang 2004] provides a sophisticated Incremental Funding Methodology that addresses that question. Teams with Delivering fluency sidestep this need, because they use evolutionary design to build their technical infrastructure incrementally.

Teams with an established product don’t always need a sophisticated plan. Rather than thinking in terms of increments or releases, these teams work from a small list of stories and constantly release minor changes. You can think of this is an adaptive plan with very short planning horizons.

Finally, adaptive planning is often seen as an alternative to predictive planning, but as “Balancing Adaptability and Predictability” on page XX shows, it’s more of a spectrum of different planning horizons. If you’re in an environment that needs predictive planning, see how many adaptive ideas you can use along with your longer planning horizons.

Further Reading

Software By Numbers [Denne and Cleland-Huang 2004] provides a compelling and detailed case for conducting frequent releases.

Agile Software Development Ecosystems [Highsmith 2002] has an excellent discussion of adaptive planning in Chapter 15.

Lean Software Development [Poppendieck and Poppendieck 2003] discusses postponing decisions and keeping your options open in Chapter 3.

XXX Consider Reinertson, The Principles of Product Development Flow: Second Generation Lean Product Development

XXX Bas Vodde: Agile Planning (Cohn)

XXX Consider David J. Bland, Testing Business Ideas, https://www.strategyzer.com/books/testing-business-ideas-david-j-bland

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.