AoAD2 Chapter 3: How to Be Agile

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020 by James Shore and Shane Warden. Do not distribute or republish without James Shore’s express written permission.

How to Be Agile

What does it mean to “be Agile?”

As we’ve seen in the previous chapters, Agile is a philosophy: a way of thinking about software development. To be Agile, you need to put the Agile philosophy into practice. Paying lip service to Agile ideas doesn’t get you anything. It’s action that matters.

Practicing Agile

Every team has a way of working—a process, or method—that they follow, even if it isn’t formally written down. It’s rarely articulated, and isn’t necessarily self-consistent, but that method reflects an underlying development philosophy.

For a team to be Agile, they need to change their method to reflect the Agile philosophy. This is both easier and harder than it sounds. It’s easy because, in most cases, they can start with one of the many off-the-shelf Agile methods, like the one in this book. It’s hard because they need to change their way of working, and that involves changing a lot of habits.

Methods consist of individual elements the Agile community calls practices. These are things like using version control, having automated builds, and giving demos to stakeholders. Most practices have been around for decades. Agile methods combine them in unique ways, accentuating those parts that support the Agile philosophy, discarding the rest, and mixing in a few new ideas. The result is a lean, powerful, self-reinforcing package.

It’s tempting to choose practices that are similar to what you’re already doing and ignore practices that are different. Avoid this temptation! The practices that are the least familiar are often the ones that involve the biggest change in philosophy. They’re the ones that your teams need the most, if they’re really going to be Agile.

In addition, Agile practices often perform double- and triple-duty, solving multiple software development problems simultaneously and supporting each other in clever and surprising ways. You won’t truly understand how it all works until you’ve seen it in action for a while.

So, although it’s tempting to customize your Agile method from the beginning, it’s best to start out with a by-the-book approach. Put it into practice—the whole thing—and spend several months refining your usage and understanding why it works. Then customize. Choose one of the rough edges, make an educated guess about how to improve, observe what happens, and repeat.

The Road to Success

The majority of this book—parts two through four—is dedicated to a curated set of Agile practices that have been proven in practice.1 To succeed with Agile, follow these steps:

1The method in this book is primarily based on Extreme Programming, but it also draws inspiration from Scrum, Kanban, Lean Software Development, the DevOps movement, and Lean Startup.

  1. Choose the zones that your organization both needs and is willing to pay for, as described in the “Choose Your Zones” section at the end of the previous chapter.

  2. Figure out which investments your organization needs to make (see the “Investing in Agility” chapter), and get your managers, teams, and stakeholders on board with making those investments, as described below.

  3. Help your teams to apply all your chosen zones’ practices together, as much as they can. The practices are self-reinforcing, so they work best when you use them all together. Parts II-IV of this book describe the practices for each zone.

  4. Follow the practices rigorously and consistently. If a practice doesn’t work, try following this book’s approach more closely. Teams new to Agile often under-apply the practices. Depending on which zones you choose, expect each team to take up to four months for the practices to start feeling comfortable and another two to six months for them to become second nature.

  5. As your teams become confident they’re applying the practices correctly—again, give it several months—they can start experimenting with changes that aren’t by the book. Each practice in this book includes a discussion of why the practice works and how it can be changed. Each time a team makes a change, they should critique what happens and make further improvements. (The team retrospective is a good place for this—see the “Retrospectives” practice.) Each team will have its own challenges and should make its own changes. Every team will end up with its own custom process.

  6. There is no last step. Agile software development is an ongoing process of learning and improvement. Never stop practicing, experimenting, and evolving.

Kaizen and Kaikaku

Kaizen (rhymes with “I win”) is a common term in the Agile community. It’s a Japanese word meaning “improvement.” In the West, it’s come to mean a process of continuous gradual improvement: constantly reviewing and improving your practices at all levels of the organization.

Continuous improvement is an integral part of being Agile, so shouldn’t you kaizen your way to Agile in the first place? Counterintuively... probably not. Kaizen is for improving your existing ways of working. If you have a document-oriented culture, kaizen will help you streamline your documents. If you have a blame-oriented culture, it will help you place blame more accurately. But it won’t help you make the leap from either to Agile.

To jump from one way of working to another, you need a different Japanese word: kaikaku (rhymes with “I rock you“). Kaikaku means “radical change.” Instead of incrementally improving your existing way of working, as kaizen does, kaikaku fundamentally changes your underlying approach.

The great Agile teams I know all started with kaikaku. They figured out what they wanted from Agile, how they were going to invest in getting those results, and went all in. This can still be done incrementally—you can change one team at a time, or one division at a time—but for any given team, the change to Agile is all-in. It’s a disruptive change, but the disruption is relatively brief: less than half a year.2

2If you have many teams, the overall time frame will be longer.

I see many more companies with mediocre Agile teams than great Agile teams. One thing the mediocre teams have in common is that they didn’t go all in. They tried to kaizen their way to Agile. It seems to work at first, but invariably stalls out. People burn out on the mismatch between Agile ideas and company values. They get tired of absorbing big new ideas. Change fatigue sets in and progress stutters to a halt after several years of effort. Ironically, the disruption from approaching Agile this way lasts much longer than the disruption from kaikaku.

If your company is new to Agile ideas, regardless of whether they already use the name, use kaikaku. It’s faster and safer. If radical change is a scary proposition, reduce risk by starting with a pilot. Choose just one team, and maybe just the Focusing fluency zone. You can use that experience to guide bigger changes in the future.

Teams that are already Agile can kaizen their way to better results within their current fluency zones. See the “Improvement” chapter for details. To move to new zones—for example, if a team is fluent in the Focusing zone, and wants to add Delivering or Optimizing fluency—kaikaku is still the best approach. New zones require new investments and major changes, and that’s best done all at once.

Successful kaikaku requires discipline and care. And it starts with...

Get Management Buy-In

If you want to be Agile, you need management support. Without it, the mismatch between your teams’ Agile practices and your organization’s non-Agile culture will cause constant friction. If you’re a manager yourself, you still need your manager to be on board, and it’s best if your peers support you, too.

1. Start with a conversation

It begins with a conversation. This is often easiest one-on-one, and you’ll be most successful if your conversations are in-person, or at least over video. Start with an influential manager you trust and recruit them as an ally. They’ll help you understand who else you need to talk to and how best to approach them.

In your conversations, starting with that first manager, talk about the challenges your organization faces with software development. Based on the benefits described in the “Choose Your Agility” chapter, talk about how you think software development could be better in your company. Don’t monopolize the conversation, though; engage your audience. Briefly share the benefits of each zone and ask which zones they think are important. Ask them why. Spend more time listening than talking.

Above all, focus on what they can get and what inaction could cause them to lose rather than pushing Agile for the sake of Agile. In fact, given the extent of misunderstandings about what Agile is, you might be better off not even mentioning the word “Agile.”

2. Get the economic buyer’s approval

Your ultimate goal is to speak to the person who has the authority to make the investments your teams need, or to have a trusted manager speak on your behalf. In sales, this person is called the “economic buyer.”

Economic buyers are often surrounded by gatekeepers who see their job as protecting the economic buyer. They’ll ask you to make your pitch to them, so they can present it to the buyer. They’re not trying to steal your idea; they’re trying to save the buyer time. Sometimes they’ll assure you that they’re the real buyer, even if they don’t actually have the authority needed.

Don’t be fooled. Although it’s helpful to get gatekeepers on your side, and often necessary, it’s not enough. Gatekeepers can’t approve the investments you need. You need to talk to the real economic buyer.

Prior to talking to the economic buyer, focus your conversations on the benefits of Agile: what’s at stake. The investments are likely to be a distraction, or even a source of concern, because the people you’re talking to won’t have the authority to make those investments.

When you finally talk to the economic buyer, your goal is to get them to agree in principle to investing in Agile. You probably won’t have a lot of time, so stay focused on the big picture. It’s also often best if you approach your meeting as a conversation, rather than a presentation with slides, but your allies will know the best approach for your specific situation.

In your conversation with the economic buyer, talk to them about what they want from their organization and how Agile will help. This works even better if a manager they trust speaks informally on your behalf.

Once the buyer is on board with Agile’s benefits, talk about the investments. Summarize the investments needed for each zone (see the “Summary of Investments” sidebar), along with how those zones map to what they want, and ask the economic buyer which of those investment/benefit tradeoffs seems most appropriate. Giving them a choice of several options, rather asking for a yes/no decision, reduces the chance that they’ll reject you outright.

Assuming the economic buyer agrees to investing in Agile in principle, or at least is worth considering further, ask for permission to create a concrete proposal. Ask what they need to see in the proposal, in general, for them to approve it. Ask for them to recommend a sponsor for you to work with, provide a date when you’ll have the proposal to them—within a day or two is best, so it’s a good idea to have a rough draft done already—and ask when you should expect to hear back.

Finally, ask for permission to follow up with them. They probably won’t get back to you when they said they would, so it’s good to be able to say, “I’m following up as you requested.”

3. Make a formal proposal

If you got this far, congratulations! You’ve passed your most important hurdle. Now you need to follow through with a proposal.

The formality of your proposal will depend on your organization. Your sponsor and other allies will help you understand how to format your proposal, help you refine it, and promote it to the economic buyer. Be prompt, polite, and persistent.

In your proposal, describe the benefits your that organization can expect to see and the investments it needs to make. Be concrete. The “Choose Your Agility” chapter describes Agile’s benefits in general; translate those benefits to your actual situation, the investments the economic buyer is willing to make, and what that realistically means for your organization.

For the investments part of your proposal, read the “Investing in Agility” chapter and translate each step into concrete requests. You may have to compromise on some investments. That chapter explains how. But avoid compromising too much. The investments are ultimately what make Agile’s benefits possible.

If this sounds like too much work...

This careful buy-in process is for when support is uncertain: when you’re working with multiple teams, asking for big investments, or working in a bureaucratic organization that’s uncomfortable about Agile ideas (even if they use the name a lot).

But sometimes, that’s not the case. Sometimes you’re just helping one small team become more Agile. If you and your manager already have the power to make the investments you need, do it!

If management thinks they’re already Agile...

Some organizations—these days, a lot of organizations—think they’re already Agile. One company told me, “We’re post-Agile!” Or you might hear, “we’re little-a agile, not big-A Agile!” But when you compare the philosophy in the “What Is Agile” chapter to how the organization acts, and the benefits in the “Choose Your Agility” chapter to what they get, you see that they’re nowhere close.

There’s nothing to be gained from arguing about terminology. If they want to say they’re agile, or post-agile, or extra-super-duper agile, let them. Instead, stay focused on the situation at hand: the challenges your teams are facing. The benefits the organization could get. The investments needed to get those benefits.

If management isn’t supportive...

If you can’t get traction with managers at first, don’t give up. Put yourself in their shoes. What does organizational success mean to them? What about personal success? How can Agile help them get what they want? If the answer is “it can’t,” then maybe Agile isn’t a good fit. Choose another approach to software development, one that properly fits your organization’s culture. Waterfall is a fine choice! Keep your documentation lightweight, your approach flexible, and your delivery horizons down to 3-6 months—shorter is better—and you’ll avoid its most common failure modes.

If you have a trusted manager you can turn to, ask for help and advice. If not, try conducting an informational interview with a manager at a company who’s been through it before. (They may try to recruit you. Win/win!) The “Change Your Organization” sidebar has more ideas.

In the early days of Agile, when it was a grass-roots movement, many teams adopted Extreme Programming (a popular Agile method) on their own, with little to no permission or support. You could try that. But I don’t recommend it. In the experience reports from teams who tried it, somebody—often the project manager—ended up having to bridge the gap between company culture and Agile philosophy. It was a thankless job and burned them out.

Some people use the Kanban method to motivate organizational change.3 Kanban wraps around existing ways of working to highlight work-in-process bottlenecks and the cost of delay. It’s easy to put in place and can motivate a more Agile approach to your work.

3Note that the Kanban method is much more than just the Kanban board some teams use for planning.

Kanban is a kaizen approach to change, so it’s slow and can only go so far, but it’s very effective and can lead to permission for kaikaku. See XXX for more information.

If nothing you do makes a difference, take a hard look at what you need for personal success. Assume the status quo won’t change, because it probably won’t. Either that’s good enough for you—and it often is—or it’s time to move to a company that better fits your needs.

Get Team Buy-In

Agile puts people first, so it shouldn’t be a surprise that you need your prospective Agile teams to agree to it as well. It is possible to force people to nod agreement with gritted teeth, but—and I speak from hard-won experience, here—that road involves a lot of employee turnover. Or fails, with a passive-aggressive whimper.

When I’m asked to help teams become Agile, I always speak to each team on their own, without managers present. (I used to include managers. No longer. That’s the “hard-won experience” part.) In your case, if you’re in a position of authority, have the teams’ coaches speak to them without you.

First, though, you’ll need to choose your prospective teams. Review the next chapter’s material about choosing teams and selecting coaches. Select a few extra candidates, just in case, because the secret to team buy-in is giving them the opportunity to refuse.

When you or the coach talk to each team, explain that their team has been chosen as a possible candidate to try Agile. I explain why their managers are interested in Agile, what benefits it will bring to the organization, and how it will affect them personally. I also explain that changing work habits can be stressful, and that they should expect a period of chaos—typically, up to three months—as everyone figures out how to make Agile work for them.

The Satir Change Model is one way of explaining how teams react to change. Steven Smith has a good article on the Satir model at https://stevenmsmith.com/ar-satir-change-model/ that includes tips for helping team members through each stage.

“If you agree,” I tell them, “I’ll ask you to follow a by-the-book approach to Agile for three months. At that point, we’ll evaluate what’s working, what’s not, and make improvements.4 After six months, you’ll have the final say on whether you continue with Agile or go back to what you have now.”

4This is a little white lie. Agile involves constant improvement, so we evaluate what’s working and what’s not within a few weeks. But we do pause for a bigger evaluation at the three month mark.

Then I open the floor for questions. Teams typically a lot of questions about the process, but invariably, one of the questions is, “What happens if we say no?” My answer is always the same: “Then it doesn’t happen.” This is important! The veto must be real. If you don’t give people the option of saying, “no,” their “yes” doesn’t mean anything. By giving people a real chance to refuse now, and a concrete opportunity to change their minds later, you make it safe to try something new.

Make sure you allow plenty of time to answer everybody’s questions. The meeting usually takes about an hour, but sometimes it goes longer. After everyone’s questions have been addressed, ask for a vote. Remind everyone that there’s no consequence for voting against Agile. Mean it. Then ask the team to vote.

If team members are skeptical...

Skepticism is normal, and you should expect it. Be straight with your teams: change is disruptive, but the results are rewarding. Be clear about the practices that you think people might find frustrating or unusual at first, such as pair programming. That will help disarm skepticism and also make it easier to introduce those practices in the future.

It helps to emphasize that this is an experiment, and that the team has the final say over whether they stick with Agile or not. I often ask the team to schedule the three-month and six-month evaluation meetings on the spot. That makes them real.

If a few team members refuse...

If a few people disagree, ask them to explain why, and see if their objections can be resolved. If they can’t, ask if they’d be willing to reserve judgement and go along with the rest of the group for six months.

If they still don’t agree, ask if they’d be okay moving to another team, subject to management approval. If that’s not okay, or if the dissenters won’t speak up (in the case of an anonymous vote), then this team isn’t a good candidate.

If the majority of the team refuses...

If the team doesn’t agree, then you’ll have to choose another team. I’ve only had this happen once, but it does happen. In my case, it was because the team didn’t trust their organization to give them the time they needed to learn. In retrospect, they were correct, and it’s a good thing we didn’t go through with the change.

If people lie about their acceptance...

Sometimes, people will vote for Agile while secretly opposing the changes. Other than making sure people don’t feel coerced, there’s nothing you can do about it. It’s not productive to second-guess people’s votes.

Even if nobody lies, the nature of change is that everyone will have second thoughts once in a while. You’ll need to work through those objections as they arise. When they do, it helps to be able to remind people that they to agreed to stick with the experiment for six months, but there is a clear end date, and if it still isn’t working by then, they’ll be able to change their mind. Be compassionate and respectful; changing habits is takes time, and people can feel like the norms they rely on have been set adrift.

In my experience, by the time the six month date rolls around, the chaos of change will be past and people will be happy with their Agile method. That’s been true for every team I’ve worked with.

If you’re introducing Agile to a lot of teams...

Changes that affect a lot of people are much more disruptive than changes that only affect a few teams. The disruption is multiplicative. Rumors start flying, people start worrying about their jobs, and the upcoming changes become part of every conversation.

Large changes—those that directly impact more than 30-70 people—require professional change management. Depending on the size of your organization, your HR department may have change management experts on staff who can help. If not, you can hire consultants. Some Agile consultants have change management experience and can guide both aspects of your Agile adoption. For very large changes, you’ll need dedicated change management experts to work alongside your Agile guides.

Organizational leaders often underestimate the importance of expert change management. This is a grave mistake! To put it in terms of the Satir Change Model, everybody goes through a period of resistance and chaos when they learn about a change. But everybody goes through it at their own pace, and leaders typically go through it first. Not because they’re better or more resilient, but because they know about it first.

By the time the rest of the organization learns about the change, organizational leaders have already experienced the “transforming idea” that resolves their feelings of resistance and chaos. To the leaders, the change now seems obvious and necessary. Why would anybody disagree?

Then they introduce the change and experience massive amounts of pushback and disruption from people going through their own resistance and chaos stages. It can kill the change entirely.

Proper change management can’t prevent all disruption, but it can reduce it. Don’t skimp. If you’re not prepared to get expert help, limit your Agile changes to a few teams at a time.

Get Stakeholder Buy-In

Your teams’ stakeholders are everyone who is affected by, or has an influence over, their work. For you and your Agile improvement efforts, your stakeholders also include anybody else who has influence over the changes you want to make.

You don’t need every stakeholder to buy in to your Agile intentions. You do need the ones with a lot of political influence to do so. If they don’t, they could quietly sabotage your efforts, pulling the rug out from under you in six months to a year, even if—or especially if—Agile is succeeding.5

5Alistair Cockburn calls this “organizational antibodies.” The more successful a change initiative is, the more people worry that it will affect them, and the more they fight against it.

The stakeholders who are most likely to resist are your teams’ business partners: product management, marketing, and sales. Agile represents a major shift in how teams interact with these groups. They’re used to a predictive approach, with a focus on commitments and deadlines. Their interactions with development teams are typically focused on documents, progress reports, and sign-offs.

Agile teams focus on feedback, frequent delivery of value, and adapting their plans. They constantly ask their stakeholders for feedback, then change their plans based on what they learn. Because their plans are always changing, they don’t make detailed release commitments. Some teams do provide release forecasts, but even then, those forecasts aren’t commitments, and they change frequently.

Some stakeholders love it. Finally, they know what’s really going on, and they have the ability to influence the outcome. Others, particularly those who have been burned by missed commitments in the past, see Agile as a political maneuver: a complicated way for teams to avoid making promises. They fight Agile tooth and nail.

Talk to your teams’ key stakeholders about Agile. This is often best done one-on-one. The topic is politically fraught, so make sure you plan your strategy with your management allies. You might not be the best person to have the conversations—a manager or person your stakeholders trust might be a better choice.

When you (or your proxy) talk to stakeholders about Agile planning, be sure to put yourself in their shoes. Demonstrate that you know what’s important to them. Perhaps they need to make marketing plans, or talk to sales prospects about competitive new features, or coordinate delivery with a third party. You want to make a connection: to dispel their preconceptions that you’re just another arrogant engineer here to tell them they can’t have what they need.

Talk about what you’re going to do better. The “Planning” chapter describes how it works. Don’t sugar-coat or over-promise. Instead, say something along the lines of, “We want to do better. We’ve found that, when we fix our plans in advance, we end up disappointing customers with delays or half-baked results. That’s not good for you or our reputation as a company. Instead of making a fixed plan and surprising you with delays, we’re going to give you more visibility and control, so you can see problems early, and we’ll adjust our plans as we go to get better results.” For many stakeholders, it’s not missed deadlines that are the biggest problem; it’s being blindsided by them.

Throughout each conversation, treat your stakeholders as trusted partners. You want them to be successful. You’ve got to balance multiple interests, and Agile teams don’t use predictive planning, but you’re there to help. You’re going to do everything you can to make their job easier and more successful.

If concrete commitments are required...

The Agile way is adaptive planning: changing your plans as you go to achieve the best possible results. That’s incompatible with making precise commitments in advance. You can commit to a specific date and steer your plans to meet that date, but you can’t predict exactly which features will be done on that date.

If that’s not good enough, fluent Delivering teams have the ability to make reliable forecasts, although it takes 4-8 weeks of development to dial them in. This allows you to predict when a particular set of features will be done. You can combine that with long planning horizons and inflexible plans to make fixed-scope, fixed-date commitments.

If your teams aren’t fluent in the Delivering zone, or if you need to make predictions before development begins, you can keep using whatever big-picture planning technique you’re using today. Agile won’t make it worse, and as development proceeds, it will give you visibility into how things are really working out. Predictive plans aren’t the Agile way, but there’s more to Agile than adaptive planning, and you can work your way up to true agility over time.

That said, predictive planning is incompatible with the Optimizing zone, which is based around taking advantage of your ability to change your plans. If you have to make commitments in advance, limit your aspirations to the Focusing and Delivering zones.

Do be careful. Some people like finding scapegoats, and some companies have a culture of assigning blame. Fixed scope, fixed date commitments are notoriously unreliable. Even if Agile doesn’t make anything worse, it might be convenient to blame Agile—and you personally—for the same deadlines that would have been missed before Agile. If that’s the case at your company, introducing Agile could be a mistake.

If stakeholders don’t buy in...

Some software teams have a contentious relationship with their stakeholders, particularly those in product management and sales. It can get pretty acrimonious. In some cases, bad blood and a lack of trust might lead stakeholders to flat out refuse to support an Agile process. They might also object to the initial slow-down involved with learning Agile (see the “Account For Learning” section in the next chapter).

If only a few stakeholders object, you can choose teams that they’re not involved with. If a lot of stakeholders object, or if their high-level leadership objects, you might be able to convince them to try out a pilot with a single team. If you can, choose a team whose stakeholders are both influential and eager to try out new ideas. It might take a while—even a year or two—but they’ll come to like the visibility and control Agile gives them, and they’ll convince their colleagues to give Agile a chance.

Sometimes software organizations try to force Agile on their stakeholders. They can even succeed, if they have enough political power, but it leads to long-term blowback. If you face widespread, active opposition, to the point that even a pilot team isn’t acceptable, Agile isn’t a good fit for your organization.

Further Reading

Once you have the buy-in you need, it’s time to make your investments. We’ll talk about that in the next chapter.

XXX Reading recommendations TBD.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.