AoAD2 Chapter 2: Choose Your Agility

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020 by James Shore and Shane Warden. Do not distribute or republish without James Shore’s express written permission.

Choose Your Agility

Real talk: There’s no point in Agile for the sake of Agile. The only reason to be Agile is to improve your results. So, what results will you get?

Let’s Talk Success

When I was a kid, I was happy to just play around with computers. I loved the challenge of programming. When I got a program to work, it was a major victory. Back then, even a program that didn’t work was a success of some sort, so long as I had fun writing it. My definition of success centered on personal rewards.

As I gained experience, my software became more complicated. I often lost track of how my programs worked, and I had to abandon some programs before they were finished. I began to believe that maintainability was the key to success—an idea that was confirmed as I entered the workforce and began working with teams of other programmers. I prided myself on producing elegant, maintainable code. Success meant technical excellence.

Despite good code, some products flopped. Even impeccably executed code could elicit yawns. I came to realize that my teams were part of a larger ecosystem involving dozens, hundreds, or even thousands of people. My software needed to satisfy those people... particularly the ones signing my paycheck. In fact, for the people funding the work, the value of the software had to exceed its cost. Success meant delivering value to the organization.

At each step along my journey, my understanding of the world expanded. For a while, I found myself perplexed by the companies that achieved massive organizational success despite creating technical messes and treating their people terribly. Now I've come to realize that my previous conclusions weren’t wrong. They were just incomplete.

  • Organizational success makes a company;

  • Lack of technical success breaks a company;

  • And personal success glues it together.

You need all three types of success to get the best results.

A Venn diagram showing three circles, marked “Organizational Success,” “Technical Success,” and “Personal Success.” There’s a star in the center where all the circles intersect.

Figure 1. Types of success

The Importance of Organizational Success

Organizational success is often neglected by software teams. They assume that, if they just do what they’re told, they’ll be successful.

Rest assured, however, that even if your teams aren’t taking responsibilty for success, the broader organization is judging them at this level. For senior management, it’s not enough for your software to be elegant, maintainable, or even beloved by its users. These are just means to an end. They care about results: the return on investment your teams produce. If your teams don’t achieve this sort of success, management will take steps to ensure that they do.

Unfortunately, senior managers don’t usually have the time or perspective to apply a nuanced solution to each team. They wield swords, not scalpels. They rightly expect their product teams to take care of fine details.

When senior management is unhappy with teams’ results, the swords come out. Costs are the most obvious target. There are two easy ways to cut them: set aggressive deadlines to reduce development time, or ship the work to a country with a lower cost of labor. Or both.

These are clumsy techniques. Aggressive deadlines end up increasing schedules rather than reducing them [Mcconnell 1999] (p.220), and offshoring has hidden costs of its own [Overby 2003].

Do aggressive deadlines and the threat of offshoring sound familiar? If so, it’s time for your teams to take back responsibility for success: not just personal or technical success, but organizational success as well.

Enter Agility

Will Agile help your teams be more successful? That depends on how much your company embraces its ideas. Agile development focuses on achieving personal, technical, and organizational successes. If you’re having trouble with any of these areas, Agile might help.

Organizational Success

Organizational success is crucial. Without it, the money will eventually run out. Organizational success doesn’t just mean dollars and cents, though; see the “What Organizations Value” sidebar.

Agile teams achieve organizational success by focusing on delivering value and decreasing costs. This directly translates to increased return on investment (ROI). Agile teams also validate expectations early, so if an idea won’t provide good ROI, you’ll find out early enough to stop working on it before your organization has spent much money.

Specifically, Agile teams increase value by seeking out business expertise, getting frequent user and customer feedback, and focusing development on the core value that their products provide. Agile teams release their most valuable features first and release updates frequently. When business needs change, Agile teams change direction to match. The best Agile teams actively seek out opportunities to adapt their plans.

Agile teams decrease costs as well. They do this partly via technical success; the best Agile teams generate only a few bugs per month. They also eliminate waste by cancelling bad products early and replacing expensive development practices with simpler ones. Agile teams communicate quickly and accurately, and they’re not bottlenecked by the work of a single team member. They regularly improve their process and code, making their software easier to maintain and enhance over time.

Technical Success

Technical success grants teams the ability to cost-effectively enhance and maintain their software. Without technical success, costs rise and your teams’ ability to make changes plummets. Eventually, they have to throw their software away and rewrite it. It can take years for the replacement to achieve feature parity with the old code. Meanwhile, they’re spending buckets of money just to catch up with where they used to be, all while their customers become increasingly frustrated with their lack of progress.

I highlight two Agile approaches to technical success in this book. The first is Extreme Programming (XP), an Agile method that’s particularly adept at technical successes. In XP, programmers work closely together, which helps them keep track of the nitpicky details necessary for great work. Programmers continuously integrate their code, which enables the team to release their software whenever it makes the most business sense. The whole team focuses on finishing each feature completely before starting the next, which prevents unexpected delays and allows the team to change direction at will.

In addition to the structure of development, XP includes advanced technical practices that lead to excellence. The most well-known practice is test-driven development, which helps programmers write code that does exactly what they intend. XP teams also create simple, ever-evolving designs that are easy to modify when plans change.

I also include techniques from the DevOps movement2, which brings XP’s ideas into the modern era of cloud-based software. Agile teams using XP and DevOps techniques create high-quality, low-defect software that’s easy to modify, maintain, and operate.

2Not to be confused with DevOps tools, which dominate much of the discussion around DevOps.

Personal Success

Personal success is, well, personal. Most people need more than a paycheck to feel satisfied, and they don’t always complain when they’re unhappy. In the best case, they find a new job, taking their accumulated organizational knowledge with them. In the worst case, they’ll quietly sabotage the company’s efforts as they agitate for work that’s more satisfying.

Agile development won’t satisfy everyone’s needs, and it’s different enough that it can be uncomfortable at first. However, once people get used to it, they find a lot to like about Agile:

Executives and senior management
People focused on the bottom line appreciate Agile teams’ emphasis on providing visibility, a solid return on investment, and long-lived code.
Users, stakeholders, domain experts, and product managers
People who care about what the software can do appreciate their ability to influence the direction of development; teams’ focus on delivering useful and valuable software; and increased delivery frequency.
Product and project managers
People interested in delivering a valuable product appreciate their ability to change direction as business needs change; teams’ reliability; and improved stakeholder satisfaction.
UX and graphic designers
People obsessed with user experience appreciate their ability to influence development plans; teams’ willingness to iterate on ideas; and faster, more accurate implementation of design ideas.
Programmers
People who write code appreciate the improved quality of life that results from technical excellence; their improved influence over forecasts and schedules; and their autonomy.
Testers
People who focus on quality appreciate being integrated with the rest of the development team; their ability to influence quality at all stages of development; and more challenging, less repetitive work.
Operations
People who keep software running in production appreciate being included in development decisions; having operational requirements included in development plans; programmers’ focus on providing actionable telemetry and logging; and teams taking responsibility for post-release quality and up-time.

Working on Agile teams has provided some of the most enjoyable moments in my career. Imagine the camaraderie of a team that works together to identify and deliver products of lasting value, with each team member enthusiastically contributing to a smooth-running whole. Imagine how it feels to take responsibility for your areas of expertise, whether technical, business, or management, with the rest of the team trusting your professional judgment and ability. Imagine how pleasant it is to address the frustrations of your work and see quality improve over time.

Agile development changes the game. Developing and delivering software in a new way takes a lot of work and thought. Yet if you do it consistently and rigorously, you’ll experience amazing things: you’ll ship truly valuable software on a regular basis. You’ll demonstrate progress multiple times per month. You’ll have the most fun you’ve ever had in software development.

The Agile Fluency® Model

I’ve painted a rosy picture of Agile. Is it too good to be true? That’s up to your company.

In 2014, I partnered with Diana Larsen to analyze why companies see such different results from their Agile teams. We’ve both worked with Agile teams from the beginning, and, over the years, we noticed that Agile teams tend to cluster in different “zones” of capability. We captured these observations in the Agile Fluency3 Model [Shore and Larsen 2018].

3“Agile Fluency” is a registered trademark of Agile Fluency Project LLC, founded by Diana Larsen and James Shore.

A picture of the Agile Fluency Model. It shows a path from “Pre-Agile” through a shift in team culture to “Focusing,” followed by a shift in team skills to “Delivering,” then a shift in organizational structure to “Optimizing,” and finally a shift in organizational culture to “Strengthening.” The path continues and fades away, as if there are additional zones yet to be discovered.

Figure 2. The Agile Fluency Model

Although the “Agile Fluency Model” figure shows a clear path from one zone to the next, reality is messier: each zone actually consists of multiple skills, all of which develop in parallel. A team has proficiency in a zone when they’re able to apply all of its skills. They have fluency when they can do so with effortless grace, even under pressure. Teams can develop fluency in any order, although the progression shown in the figure is most common.

The zones correspond to how companies invest in Agile ideas. In other words, the fluency your teams achieve—and the results your company gets—depends on how much it buys in to the Agile philosophy. Not just lip service, but actual, meaningful changes that cost time, money, and political capital.

Each zone has its own set of investments, and you probably won’t be able to convince your company to invest in every zone. That’s okay. Each zone also has its own set of benefits, and as long as your company fully invests in at least one zone, you’re likely to see improvements.

Which zones should you choose? It’s a matter of picking the right cost/benefit tradeoffs.

Focusing Zone

The Focusing zone corresponds to the fundamentals of Agile. For your teams to reap the benefits of Focusing fluency, your organization needs to buy in to the core Agile philosophy.

What is that core philosophy? As we saw in the “Essence of Agile” section: Agile is adaptive rather than predictive, and people-oriented rather than process-oriented.

In practice, this means that Focusing teams regularly review and change their plans. They work as a tightly-knit team to define their own work processes and self-organize around completing the work. All the while, they’re Focusing on the next piece of value for their organization.

For most teams and organizations, this requires a shift in how they think about teams. Pre-Agile organizations make plans in advance, ask teams for estimates, and expect reports about how work is progressing relative to those estimates. Focusing teams revise their plans frequently—at least every month—and demonstrate progress by showing what they’ve completed.

Pre-Agile organizations break their plans into tasks, assign those tasks to individuals on the team, and judge individuals based on how well they complete their tasks. Focusing teams do their own task breakdown, decide for themselves who will work on each task, and expect to be judged on their ability to create value as a team.

Can your organization buy into this core Agile philosophy? Again, lip service isn’t enough. They’ll need to make concrete investments in the form of changes to team structure, management, and work environment. (See the “Investing in Agility” chapter for details.) It’s a “good news, bad news” situation: The bad news is that, when the rubber meets the road, some organizations won’t be willing to invest. The good news is that, if they refuse, you’ve discovered early that they’re not really on board with the Agile philosophy. You just saved yourself years of frustration and heartache chasing Cargo Cult Agile.

If you are able to get buy-in, Focusing fluency will take each team about 2-6 months of dedicated practice to develop, with benefits appearing within 1-4 months. Part II shows how. In exchange, expect to see the improvements in value described in the “Organizational Success” section. Some aspects of personal success described in the “Personal Success” section will also improve. Specifically, people with a business and user focus will like their improved visibility into the team’s work and their ability to influence teams’ plans.

You aren’t likely to see the improved technical success described in the “Technical Success” section, though, or the organizational cost savings that come with it. In fact, technical quality could get worse. Traditional approaches to design rely on careful up-front work based on firm plans. That doesn’t mesh well with Agile’s emphasis on flexible, frequently changing plans. As a result, Focusing teams often end up with an ad-hoc approach to design, which isn’t sustainable.

Poor technical quality can be frustrating for team members, lowering their feelings of personal success. It usually prevents the team from making reliable forecasts and commitments, which can be frustrating for stakeholders.

To improve technical success and make useful forecasts, you’ll also need fluency in the Delivering zone.

Delivering Zone

Companies who invest in the Focusing zone have taken an important first step: they’ve aligned themselves with the core Agile philosophy. This is critical! It’s the secret to Agile success.

But it’s not enough for long-term success. Although organizational success makes a company, and Focusing fluency will help you get it, lack of technical success breaks a company. Software built by Focusing teams becomes unmaintainable over time. Over the course of several years, costs will rise. The team will eventually lose their ability to make cost-effective changes. They’ll tell you they need to throw away the software and rewrite. That can kill a product.

Delivering teams prevent this problem by investing in technical success. They design their code so that it can absorb frequent changes. They keep code quality high, so they don’t waste time hunting bugs. They refine their production lifecycle so releases are painless and operations are manageable. They’re capable of Delivering reliable, low-defect software whenever it makes the most business sense.

This requires a shift in team members’ skills, which requires a substantial investment in learning and code quality, as well as structural changes to integrate technical disciplines such as Testing and Ops into each team. (See the “Investing in Agility” chapter for details.)

If your company makes these investments, Delivering fluency will take each team 3-24 months to develop, in addition to the time needed for Focusing fluency, although you’ll see benefits within 2-6 months. Part III has the techniques. The exact amount of time each team will need depends on the quality of their existing code and how much coaching they receive.

In exchange, expect to see the technical success factors described in the “Technical Success” section, the cost reductions described in the “Organizational Success” section, and the team member success factors described in the “Personal Success” section. Combined with Focusing fluency, that’s all the success factors described at the beginning of this chapter.

Optimizing Zone

The vast majority of companies would be satisfied with Focusing and Delivering fluency. But Agile imagines more. In its full glory, Agile is a world in which teams twirl and dance in response to changing market conditions. They experiment and learn; develop new markets; outmaneuver the competition.

Optimizing teams achieve this level of agility. They understand what their market wants, what your business needs, and how to bridge the two. Or, as in a startup environment, they know what they need to learn and how to go about learning it. They’re constantly Optimizing their product plans to achieve the most value possible.

This requires a shift in organizational structure. Creating optimal plans requires constant attention by people with deep business and product expertise, and that means having product and market experts join development teams full-time. It also means giving those teams full responsibility for their product budgets and plans.

These structural changes require high-level permission in the organization. It can be difficult to obtain. It typically takes teams at least a year of building trust via Delivering fluency before they get permission for these investments. Once that happens, Optimizing fluency takes another 3-9 months to develop, although benefits appear within 1-3 months. That said, Optimizing is a never-ending process of experimentation, learning, and discovery. Part IV describes how to begin.

Strengthening Zone

There’s one final zone in the Agile Fluency Model. It’s largely speculative: a possible future for Agile. It’s also only appropriate for organizations on the bleeding edge of management theory and practice. That puts it out of scope for this book. Briefly, the Strengthening zone involves distilling teams’ collective insights and channeling them into organizational improvements. If you’d like to learn more, start with [Shore and Larsen 2018].

Choose Your Zones

Will Agile get you better results than you’re seeing now? It depends on which zones your organization can support. In a vacuum, the first three zones together provide the best results and the purest realization of Agile ideas. But that also takes the most investment. Investing in more than you need could incur backlash. Only invest in the zones that will be a big benefit for your organization.

Once you’ve chosen a zone, though, invest fully in it. Without sufficient investment in a zone, the skills teams need to become fluent in that zone will develop slowly, if at all, and full fluency will probably be out of reach. You’ll incur the costs of learning without getting all the benefits, and you might even see worse results than now if teams don’t have the support they need.

In other words, choose the zones that your company both needs and is willing to pay for.

So, which zones should you choose? As you’ll see in the next chapter, it starts with a conversation with management. But here’s a handy guide.

  • Every Agile team needs Focusing fluency. It’s fundamental. If your company can’t at least invest in Focusing fluency, Agile isn’t a good fit. You don’t need perfect support, as the “Investing in Agility” chapter describes, but if you can’t at least get the minimum, you’ll need a different approach.

  • Delivering fluency decreases costs and increases development speed. It takes time to learn, but you’ll see benefits within 2-6 months, and it will pay for itself within a year. Without it, your code will eventually succumb to technical debt. That makes the Delivering zone a no-brainer for most teams. That said, some organizations aren’t ready to make the big investments in learning and code quality that the Delivering zone requires. It’s okay to start with Focusing fluency first, demonstrate success, and then use that to make further investment.

  • Optimizing fluency is where Agile shines brightest. It’s also a lot to ask. For most organizations, it’s best to build trust by demonstrating fluency in the Delivering zone first, then gradually take on more responsibility. But if your organization already has a culture of delegating decision-making authority to cross-functional teams, as is often seen in startups, Optimizing fluency will give you great results.

Whichever zones you choose, invest in learning all of them simultaneously. It’s fastest and easiest to learn Agile ideas if you practice everything together, because the techniques in the later zones make the earlier zones work better. If you can’t get all the investments you want, though, that’s okay. It takes longer, but as long as you can at least invest in Focusing, you can build up to the later zones over time.

Once you’ve chosen your zones, you’re ready to get started. That’s the topic of our next chapter.

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.