AoAD2 Chapter 6: Invest in Change

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Invest in Change

You’ve decided that Agile will make your teams more successful. You know which zones have the best cost/benefit tradeoffs. You’ve figured out which investments your company needs to make. Now, how do you make it happen?

Understanding Change

A chart with two axes. The horizontal axis is labelled “Time,” with an arrow pointing to the right. The vertical axis is labelled “Performance,” with an arrow pointing up. Within the chart, a line travels from left to right. The line starts in the middle of the performance axis with a section labelled “late status quo.” It is mostly horizontal, with small vertical pertubations and a slight upward trend. Next, there’s a star labelled “foreign element.” The line continues as before through a short section labelled “resistance,” before plunging down (indicating lowered performance) with dramatic up-and-down spikes (indicating highly-varying performance]. This section is labelled “chaos.” None of the spikes exceed the midline performance of the late status quo. About halfway along the horizontal axis, one of the spikes is labelled with another star and the title “transforming idea.” The spikes gradually climb, indicating increased performance, and the size of the swings decrease. This section is labelled “integration.” Finally, the spikes flatten out, at a higher level of performance than the beginning, with small vertical pertubations and a slight upward trend. This final section is labelled “new status quo.”

Figure 1. The Satir Change Model

Change is disruptive, and moving to Agile is no exception.

Change is disruptive, and moving to Agile is no exception. Exactly how disruptive it is depends on how many teams are affected and how well you manage the change. If you have one team that’s eager to try Agile with your organization’s full support, it doesn’t have to be a big deal. If you’re trying to change 50 teams in an organization that’s unfamiliar with Agile ideas... well, now it’s a very big deal.

One way to understand how people respond to change is Virginia Satir’s Change Model, shown in figure “Satir Change Model”.1 As the figure shows, there are five stages to a change. Here’s how they apply to Agile:

1Steven Smith has a good article on the Satir model at https://stevenmsmith.com/ar-satir-change-model/ that includes tips for helping team members through each stage.

  1. Late Status Quo. This is the pre-Agile way of working. It’s comfortable and familiar. Everyone knows what’s expected of them and how to do their job. Some people aren’t completely happy, though, and they think Agile will help. They push for change.

  2. Resistance. The people who want change start getting traction, and some sort of Agile change becomes likely. This is called a “foreign element,” and people start responding to the possibility of change. Many will oppose it. They’ll say changing to Agile is unnecessary, unlikely to succeed, or a waste of time. Some will be angry. The more people that are affected, the more resistance you’ll see.

  3. Chaos. The Agile change is approved and teams start using Agile practices. Old ways of working and familiar expectations no longer apply. People feel lost and confused, and moods are volatile. Some days are good; some are bad. People occasionally revert to childish behavior. Performance and morale decline.

  4. Integration. With practice, people start to become familiar with their new ways of working. They discover an aspect of Agile—called a “transforming idea”—that is particularly compelling to them. (This will be different for each person.) They embrace the possibilities Agile brings and start putting real effort into making Agile work. Feelings of chaos decrease, morale improves, and performance climbs.

  5. New Status Quo. People have made their way through the change and come out the other side. Their new Agile ways of working are comfortable and familiar, and they’re confident enough to continue to make small changes. Performance has stabilized at a higher level than before the change, and continues to increase gradually as people experiment with further small changes.

Trying to rush change just makes things worse.

This reaction to change is unavoidable. Trying to rush it just makes things worse. That’s why organizations need to set aside time for learning Agile (see “Make Time for Learning” on p.XX). Note the parallel between the Satir model shown in figure “Satir Change Model” and the J-curve shown in figure “Agile Performance Over Time”.

Everybody goes through these stages at their own pace. The length of the change, and the depth of the chaos, depends on how much their day-to-day life is affected. Someone who’s only peripherally involved will respond less than a person who’s part of a newly-Agile team. Individual personalities matter, too; some people love trying new things, while others want stability and predictability.

You can decrease (but not eliminate!) the chaos by using a technique I learned from Diana Larsen: Support, Information, and Structure (SIS).2

2Thanks to Diana Larsen for assisting with this list.

  • Support. Help people understand how to do their job in the changed environment. Provide training, coaching, and other ways for people to get help without feeling judged. Make the investments described in chapter “Invest in Agility”. Make sure everyone has someone they can talk to, either at work or in their personal life, when they’re feeling overwhelmed.

  • Information. Be transparent about what’s happening, what’s known, and what’s yet to be determined. Address people’s career concerns. If you can do so honestly, make an explicit promise that no one’s getting fired as a result of the change. Communicate more than you think should be needed.3

  • Structure. People need ground to stand on, so provide a roadmap for the change. If you’re using this book as the basis for your change, provide copies, and tell people which parts you’re using. When things are uncertain, describe what you need to do to make them certain, and when you expect that to happen. If there’s an interim step, such as temporary teams, be clear that it’s temporary, and describe what’s going to happen next.

3Diana says, “Communciate until you want to throw up. And then more.”

Large-Scale Change

Changes that affect a lot of people are much more disruptive than changes that only affect a few teams. The disruption is multiplicative. Rumors start flying, people start worrying about their jobs, and the upcoming changes become part of every conversation.

Large changes—those that directly impact more than 30-70 people—require professional change management. Depending on the size of your organization, your HR department may have change management staff who can help. If not, you can hire consultants. Some Agile consultants have change management experience and can guide both aspects of your Agile adoption. For very large changes that directly impact more than 200-500 people, you’ll need dedicated change management experts to work alongside your Agile guides.

Organizational leaders often underestimate the importance of change management. This is a grave mistake! To put it in terms of the Satir model, everybody goes through a period of resistance and chaos when they learn about a change. But everybody goes through it at their own pace, and leaders typically go through it first. Not because they’re better or more resilient, but because they know about it first.

By the time the rest of the organization learns about the change, organizational leaders have already experienced the “transforming idea” that resolves their feelings of resistance and chaos. To the leaders, the change now seems obvious and necessary. Why would anybody disagree?

Then they introduce the change and experience massive amounts of pushback and disruption from people going through their own resistance and chaos stages. It can kill the change entirely.

Don’t skimp on proper change management.

Proper change management can’t prevent all disruption, but it can reduce it. Don’t skimp. If you’re not prepared to get expert help, limit your Agile changes to a few teams at a time.

Making Changes

Kaizen (rhymes with “I win”) is a common term in the Agile community. It’s a Japanese word meaning “improvement.” In the Agile community, it specifically means continuous, gradual improvement.4

4Kaizen was imported to Agile from Lean Manufacturing, which itself is based on the revolutionary Toyota Production System—hence the Japanese terminology.

Continuous improvement is an integral part of Agile, so shouldn’t you kaizen your way to Agile in the first place? Counterintuively... probably not. Kaizen is for improving your existing ways of working. If you have a document-oriented culture, kaizen will help you streamline your documents. If you have a blame-oriented culture, it will help you place blame more accurately. But it won’t help you make the leap from either to Agile.

To jump from one way of working to another, you need a different Japanese word: kaikaku (rhymes with “I rock you“). Kaikaku means “transformative change.” Instead of incrementally improving your existing way of working, as kaizen does, kaikaku fundamentally changes your underlying approach.

The great Agile teams I know all started with kaikaku. They figured out what they wanted from Agile, how they were going to invest in getting those results, and they went all in.

I see many more mediocre Agile teams than great Agile teams. One thing the mediocre teams have in common is that their companies didn’t go all in. They tried to kaizen their way to Agile. It seems to work at first, but invariably stalls out. People burn out on the mismatch between Agile ideas and company values. They get tired of absorbing new ideas. Change fatigue sets in and progress stutters to a halt after several years of effort. Ironically, the disruption from approaching Agile this way lasts much longer than the disruption from kaikaku.

If your company is new to Agile ideas—regardless of whether they already use the name—use kaikaku. Choose your zones, make the investments, and have each team start using all the corresponding practices at once. It may seem scary, but it’s actually faster and safer than adopting practices gradually.

If you’d like to take an incremental approach, kaikaku is still your best bet. Rather than using kaizen to gradually introduce Agile to a lot of teams, use kaikaku to fully introduce Agile to one team. For an even smaller increment, start with the Focusing zone alone. Then add the Delivering zone. Then add another team, perhaps with Focusing and Delivering together. As you gain experience, you can increase the size of your increments.

Teams that are already Agile can kaizen their way to better results within their current fluency zones. See chapter “Improvement” for details. To move to new zones—for example, if a team is fluent in the Focusing zone, and wants to add Delivering or Optimizing fluency—kaikaku is still the best approach. New zones require new investments and major changes, and that’s best done all at once.

Successful kaikaku requires discipline and care. And it starts with...

Get Management Buy-In

Agile requires management support. Without it, the mismatch between your teams’ Agile practices and your organization’s non-Agile culture will cause constant friction. If you’re a manager yourself, you still need your manager to be on board, and it’s best if your peers support you, too.

1. Start with a conversation

It begins with a conversation. This is often easiest one-on-one, and you’ll be most successful if your conversations are in-person, or at least over video. Start with an influential manager you trust and recruit them as an ally. They’ll help you understand who else you need to talk to and how best to approach them.

In your conversations, starting with that first manager, talk about the challenges your organization faces with software development. Based on the benefits described in chapter “Choose Your Agility”, talk about how you think software development could be better in your company. Don’t monopolize the conversation, though; engage your audience. Briefly share the benefits of each zone and ask which zones they think are important. Ask them why. Spend more time listening than talking.

Above all, focus on what they can get and what inaction could cause them to lose rather than pushing Agile for the sake of Agile. In fact, given the extent of misunderstandings about what Agile is, you might be better off not even mentioning the word “Agile.”

2. Get the economic buyer’s approval

Your ultimate goal is for you to speak to the person who has the authority to make the investments your teams need. In sales, this person is called the “economic buyer.”

Economic buyers are often surrounded by gatekeepers who see their job as protecting the economic buyer. They’ll ask you to make your pitch to them, so they can present it to the buyer. They’re not trying to steal your idea; they’re trying to save the buyer time. Sometimes they’ll assure you that they’re the real buyer, even if they don’t actually have the authority needed.

Don’t be fooled. Although it’s helpful to get gatekeepers on your side, and often necessary, it’s not enough. Gatekeepers can’t approve the investments you need. You need to talk to the real economic buyer.

Prior to talking to the economic buyer, focus your conversations on the benefits of Agile: what’s at stake. The investments are likely to be a distraction, or even a source of concern, because the people you’re talking to won’t have the authority to make those investments.

When you finally talk to the economic buyer, your goal is to get them to agree in principle to investing in Agile. You probably won’t have a lot of time, so stay focused on the big picture. It’s also often best if you approach your meeting as a conversation, rather than a presentation with slides, but your allies will know the best approach for your specific situation.

In your conversation with the economic buyer, talk to them about what they want from their organization and how Agile will help. This works even better if a manager they trust speaks informally on your behalf.

Once the buyer is on board with Agile’s benefits, talk about the investments. Don’t overwhelm them with detail; just summarize a few key investments needed for each zone (“Summary of Investments” on p.XX will help you prepare), along with how those zones map to what the economic buyer wants, and ask them which investment/benefit tradeoffs seem most appropriate. Giving them a choice of several options, rather asking for a yes/no decision, reduces the chance that they’ll reject you outright.

Assuming the economic buyer agrees in principle to investing in Agile, or that it’s at least worth considering further, ask for permission to create a concrete proposal. Ask what they need to see in the proposal, in general, for them to approve it. Ask for them to recommend a sponsor for you to work with, provide a date when you’ll have the proposal to them—within a day or two is best, so it’s a good idea to have a rough draft done already—and ask when you should expect to hear back.

Finally, ask for permission to follow up with them. They probably won’t get back to you when they said they would, so it’s good to be able to say, “I’m following up as you requested.”

3. Make a formal proposal

If you got this far, congratulations! You’ve passed your most important hurdle. Now you need to follow through with a proposal.

The formality of your proposal will depend on your organization. Your sponsor and other allies will help you understand how to format your proposal, help you refine it, and promote it to the economic buyer. Be prompt, polite, and persistent.

In your proposal, describe the benefits your that organization can expect to see and the investments it needs to make. Be concrete. Chapter “Why Agile” describes Agile’s benefits in general, and the beginnings of parts 2-4 go into more detail. Translate those benefits to your actual situation, the investments the economic buyer is willing to make, and what that realistically means for your organization.

For the investments part of your proposal, read chapter “Invest in Agility” and translate each step into concrete requests. You may have to compromise on some investments. That chapter explains how. But avoid compromising too much. The investments are ultimately what make Agile’s benefits possible.

If this sounds like too much work...

This careful buy-in process is for when support is uncertain: when you’re working with multiple teams, asking for big investments, or working in a bureaucratic organization that’s uncomfortable about Agile ideas (even if they use the name a lot).

But sometimes, that’s not the case. Sometimes you’re just helping one small team become more Agile. If you and your manager already have the power to make the investments you need, do it!

If management thinks they’re already Agile...

Some organizations—these days, a lot of organizations—think they’re already Agile. One company told me, “We’re post-Agile!” Or you might hear, “we’re little-a agile, not big-A Agile!” But when you compare the philosophy in chapter “What Is Agile” to how the organization acts, and the benefits in chapter “Why Agile” to what they get, you see that they’re nowhere close.

Stay focused on the situation at hand: challenges, benefits, and investments.

There’s nothing to be gained from arguing about terminology. If they want to say they’re agile, or post-agile, or extra-super-duper agile, let them. Instead, stay focused on the situation at hand: the challenges your teams are facing. The benefits the organization could get. The investments needed to get those benefits.

If management isn’t supportive...

If you can’t get traction with managers at first, don’t give up. Put yourself in their shoes. What does organizational success mean to them? What about personal success? How can Agile help them get what they want? If the answer is “it can’t,” then maybe Agile isn’t a good fit. Choose another approach to software development, one that better fits your organization’s culture. See “Succeeding With Waterfall” on p.XX for one possibility.

If you have a trusted manager you can turn to, ask for help and advice. If not, try conducting an informational interview with a manager at a company who’s been through it before. (They may try to recruit you. Win/win!) “Change Your Organization” on p.XX has more ideas.

In the early days of Agile, when it was a grass-roots movement, many teams adopted Extreme Programming on their own, with little to no permission or support. You could try that. But I don’t recommend it. In the experience reports from teams who tried it, somebody—often the project manager—ended up having to bridge the gap between company culture and Agile philosophy. It was a thankless job and burned them out.

Some people use the Kanban method to motivate organizational change.5 Kanban wraps around existing ways of working to highlight work-in-process bottlenecks and the cost of delay. It’s easy to put in place and can motivate a more Agile approach to your work.

5Note that the Kanban method is much more than just the Kanban board some teams use for planning.

Kanban is a kaizen approach to change, so it’s slow and can only go so far, but it’s very effective and can lead to permission for kaikaku. See XXX [choose book recommendation] for more information.

If nothing you do makes a difference, take a hard look at what you need for personal success. Assume the status quo won’t change, because it probably won’t. Either that’s good enough for you—and it often is—or it’s time to move to a company that better fits your needs.

Get Team Buy-In

Agile puts people first, so it shouldn’t be a surprise that you need your prospective Agile teams to agree to trying Agile. It is possible to force people to nod agreement with gritted teeth, but—and I speak from hard-won experience, here—that road involves a lot of employee turnover. Or it fails, with a passive-aggressive whimper.

When I’m asked to help teams become Agile, I always speak to each team on their own, without managers present. You want them to be comfortable expressing themselves without fear of retribution. Include their coach, and if you’re a manager yourself, start the meeting by expressing your support for their decision—whatever it may be—then let them speak to the coach without you.

When you or the coach talk to each team, explain that their team has been chosen as a possible candidate to try Agile. I explain why their managers are interested in Agile, what benefits it will bring to the organization, and how it will affect them personally. I also explain that changing work habits can be stressful, and that they should expect a period of chaos—typically, up to three months—as everyone figures out how to make Agile work for them. I often sketch out and describe the Satir model (see figure “Satir Change Model”).

“If you agree,” I tell them, “I’ll ask you to follow a by-the-book approach to Agile for three months. At that point, we’ll evaluate what’s working, what’s not, and make improvements.6 After six months, you’ll have the final say on whether you continue with Agile or go back to what you have now.”

6This is a little white lie. Agile involves constant improvement, so we evaluate what’s working and what’s not within a few weeks. But we do pause for a bigger evaluation at the three month mark.

Then I open the floor for questions. Teams typically a lot of questions about the process, but invariably, one of the questions is, “What happens if we say no?” My answer is always the same: “Then it doesn’t happen.” This is important! The veto must be real. If you don’t give people the option of saying, “no,” their “yes” doesn’t mean anything. By giving people a real chance to refuse now, and a concrete opportunity to change their minds later, you make it safe to try something new.

Make sure you allow plenty of time to answer everybody’s questions. The meeting usually takes about an hour, but sometimes it goes longer. After everyone’s questions have been addressed, remind everyone that there’s no consequence for voting against Agile. Mean it. Then ask the team to vote.

If team members are skeptical...

Skepticism is normal, and you should expect it. Be straight with your teams: change is disruptive, but the results are rewarding. Be clear about the practices that you think people might find frustrating or unusual at first, such as pair programming. That will help disarm skepticism and also make it easier to introduce those practices in the future.

It helps to emphasize that this is an experiment, and that the team has the final say over whether they stick with Agile or not.

If a few team members refuse...

If a few people disagree, ask them to explain why, and see if their objections can be resolved. If they can’t, ask if they’d be willing to reserve judgement and go along with the rest of the group for six months.

If they still don’t agree, consider asking if they’d be okay moving to another team, subject to management approval. If that’s not an option, or if you don’t know who disagreed (in the case of an anonymous vote), then this team isn’t a good candidate.

If the majority of the team refuses...

If the team doesn’t agree, then you’ll have to choose another team. I’ve only had this happen rarely, but it does happen. In one case, it was because the team didn’t trust their organization to give them the time they needed to learn. In hindsight, they were right, and it’s a good thing we didn’t go through with the change.

If people lie about their acceptance...

Sometimes, people will vote for Agile while secretly opposing the changes. Other than making sure people don’t feel coerced, there’s nothing you can do about it. It’s not productive to second-guess people’s votes.

The nature of change is that everyone has second thoughts once in a while.

Even if nobody lies, the nature of change is that everyone will have second thoughts once in a while. You’ll need to work through those objections as they arise. When they do, it helps to be able to remind people that they agreed to stick with the experiment for six months, but there is a clear end date, and if it still isn’t working by then, they’ll be able to change their mind. Be compassionate and respectful; changing habits is takes time, and people can feel like the norms they rely on have been set adrift.

In my experience, by the time the six month date rolls around, the chaos of change will be past and people will be happy with their Agile method. That’s been true for every team I’ve worked with.

Get Stakeholder Buy-In

Your teams’ stakeholders are everyone who is affected by, or has an influence over, their work. For you and your Agile improvement efforts, your stakeholders also include anybody else who has influence over the changes you want to make.

We’ve already talked about team member and manager buy-in. You don’t need all your remaining stakeholders to buy in to your Agile intentions, but you do need the ones with a lot of political influence to do so. If they don’t, they could quietly sabotage your efforts, pulling the rug out from under you in six months to a year, even if—or especially if—Agile is succeeding.7

7Alistair Cockburn calls this “organizational antibodies.” The more successful a change initiative is, the more people worry that it will affect them, and the more they fight against it.

The stakeholders who are most likely to resist are your teams’ business partners: product management, marketing, and sales. Agile represents a major shift in how teams interact with these groups. They’re used to a predictive approach, with a focus on commitments and deadlines. Their interactions with development teams are typically focused on documents, progress reports, and sign-offs.

Agile teams focus on feedback, frequent delivery of value, and adapting their plans. They constantly ask their stakeholders for feedback, then change their plans based on what they learn. Because their plans are always changing, they don’t make detailed release commitments. Some teams do provide release forecasts, but even then, those forecasts aren’t commitments, and they change frequently.

Some stakeholders love it. Finally, they know what’s really going on, and they have the ability to influence the outcome. Others, particularly those who have been burned by missed commitments in the past, see Agile as a political maneuver: a complicated way for teams to avoid making promises. They fight Agile tooth and nail.

Talk to your teams’ key stakeholders about Agile. This is often best done one-on-one. The topic can be politically fraught, so make sure you plan your strategy with your management allies. You might not be the best person to have the conversations—a manager or person your stakeholders trust might be a better choice.

When you (or your proxy) talk to stakeholders about Agile planning, be sure to put yourself in their shoes. Demonstrate that you know what’s important to them. Perhaps they need to make marketing plans, or talk to sales prospects about competitive new features, or coordinate delivery with a third party. You want to make a connection: to dispel their preconceptions that you’re just another arrogant engineer here to tell them they can’t have what they need.

Talk about what you’re going to do better. Chapter “Planning” describes how it works. Don’t sugar-coat or over-promise. Instead, say something along the lines of, “We’ve found that, when we fix our plans in advance, we end up disappointing customers with delays or half-baked results. That’s not good for you or our reputation as a company. Instead of making a fixed plan and surprising you with delays, we’re going to give you more visibility and control, so you can see problems early, and we’ll adjust our plans as we go to get better results.” For many stakeholders, it’s not missed deadlines that are the biggest problem; it’s being blindsided by them.

Treat your stakeholders as trusted partners. You want them to be successful.

Throughout each conversation, treat your stakeholders as trusted partners. You want them to be successful. You’ve got to balance multiple interests, and you’re providing visibility and control, not predictability, but you’re there to help. You’re going to do everything you can to make their job easier and more successful.

If concrete commitments are required...

The Agile way is adaptive planning: changing your plans as you go to achieve the best possible results. That’s incompatible with making precise commitments in advance. You can commit to a specific date and steer your plans to meet that date, but you can’t predict exactly which features will be done on that date.

If that’s not good enough, you can use the approach described in “If Waterfall Governance Is Required” on p.XX to make fixed-date, fixed-scope plans. They’re not guaranteed to be correct, but they’ll be at least as good as what you have today. If that’s still not enough, Agile isn’t a good fit.

If stakeholders don’t buy in...

Some software teams have a contentious relationship with their stakeholders, particularly those in product management and sales. It can get pretty acrimonious. In some cases, bad blood and a lack of trust might lead stakeholders to flat out refuse to support an Agile process. They might also object to the initial slow-down involved with learning Agile (see “Make Time for Learning” on p.XX).

If only a few stakeholders object, you can choose teams that they’re not involved with. If a lot of stakeholders object, or if their high-level leadership objects, you might be able to convince them to try out a pilot with a single team. If you can, choose a team whose stakeholders are both influential and eager to try out new ideas. It might take a while—even a year or two—but they’ll come to like the visibility and control Agile gives them, and they’ll convince their colleagues to give Agile a chance.

Sometimes software organizations try to force Agile on their stakeholders. They can even succeed, if they have enough political power, but it leads to long-term blowback. If you face widespread, active opposition, to the point that even a pilot team isn’t acceptable, Agile isn’t a good fit for your organization. See “Succeeding With Waterfall” on p.XX for another option.

Further Reading

XXX Reading recommendations TBD.

Consider:

  • Agile Conversations

  • Agile Adoption Patterns (Elssamadisy)

Thomas Owens recommends:

  • Getting to Yes

  • Getting Past No

  • Crucial Conversations

Jason Yip recommends:

  • 7 Rules for Positive, Productive Change

  • Scaling Up Excellence

  • Influencer

  • https://medium.com/@jchyip/my-preferred-models-and-strategies-for-facilitating-large-scale-change-a0e498169e67

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.