The Art of Agile Development: Is XP Right For Us?

The second edition is now available! The Art of Agile Development has been completely revised and updated with all new material. Visit the Second Edition page for more information, or buy it on Amazon.

in 99 words

You'll need support, from managers and colleagues. Work together, in the same room, and include business experts. Keep the team small--five to twenty people. Don't ignore practices: they're more important than they seem.

A brand-new codebase and a language that's easy to refactor are best for learning. Try to include an experienced coach, and an experienced designer. It's best if everyone gets along.

You don't have to do any of these if you don't want to. (We provide alternatives.) But you'll have more success, and more fun, if you fix your environment rather than compromising your work.

as haiku

slippery, muddy--
where grass died, pepperbushes
provide fragrant blooms

Behind the Scenes

Truth or Clarity?

Poster

'Pre-Flight Checklist' poster

Download this poster!

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

Is XP Right For Us?

You can adopt XP in a wide variety of conditions, although the practices you use will vary depending on your situation. The practices in this book were chosen to give you the greatest chance of success. That leads to some prerequisites and recommendations about your team's environment. You don't have to meet these criteria exactly, but it's worth trying to change your environment so that you do. That will give you the best chance of succeeding.

As Martin Fowler said:1

1http://martinfowler.com/bliki/EnterpriseRails.html.

In this sense I see a startling parallel between DHH [David Heinemeier Hansson, creator of Ruby on Rails] and Kent Beck. For either of them, if you present them with a constrained world, they'll look at constraints we take for granted, consider them to be unessential, and create a world without them. I don't have that quality, I tend to try to work within the constraints gradually pushing at them, while they just stick some intellectual dynamite under them and move on. That's why they can create things like Extreme Programming and Rails which really give the industry a jolt.

In other words, if your organization puts a barrier between your work and success, don't just put up with it... find a way to remove it. It's your best path to success.

Similarly, if you want to practice XP, do everything you can to meet the following prerequisites and recommendations. It's a lot more effective than working around limitations.

Prerequisite #1: Management Support

It's very difficult to use XP in the face of opposition from management. Active support is best. To practice XP as described in this book, you will need the following:

  • A common workspace with pairing stations (see Sit Together in Chapter 6)

  • Team members solely allocated to the XP project (see The XP Team in Chapter 3)

  • A product manager, on-site customers, and integrated testers (also discussed in The XP Team in Chapter 3)

You will often need management's help to get the previous three items. In addition, the more management provides the following things, the better:

  • Team authority over the entire development process, including builds, database schema, and version control

  • Compensation and review practices that are compatible with team-based effort

  • Acceptance of new ways of demonstrating progress and showing results (see Reporting in Chapter 6)

  • Patience with lowered productivity while the team learns

If Management Isn't Supportive...

In order for management to support your adoption of XP, they need to believe in its benefits. Think about what the decision-makers care about. What does an organizational success mean to your management? What does a personal success mean? How will adopting XP help them achieve those successes? What are the risks of trying XP, how will you mitigate those risks, and what makes XP worth the risks? Talk in terms of your managers' idea of success, not your own success.

If you have a trusted manager you can turn to, ask for her help and advice. If not, talk to your mentor (see "Find a Mentor" in Chapter 2). Fearless Change: Patterns for Introducing New Ideas [Manns & Rising] is another good resource.

If management refuses your overtures, then XP probably isn't appropriate for your team. You may be able to demonstrate XP's value incrementally by adopting some standalone practices (see "Extremeties: Applying Bits and Pieces of XP," later in this chapter).

Prerequisite #2: Team Agreement

Just as important as management support is the team's agreement to use XP. If team members don't want to use XP, it's not likely to work. XP assumes good faith on the part of team members—there's no way to force the process on somebody who's resisting it.

If People Resist...

It's never a good idea to force someone to practice XP against his will. In the best case, he'll find some way to leave the team, quitting if necessary. In the worst case, he'll remain on the team and silently sabotage your efforts.

Reluctant skeptics are okay. If somebody says, "I don't want to practice XP, but I see that the rest of you do, so I'll give it a fair chance for a few months," that's fine. She may end up liking it. If not, after a few months have gone by, you'll have a better idea of what you can do to meet the whole team's needs.

One way to help people agree to try XP is to promise to revisit the decision on a specific date. (Allow two or three months if you can.) At that point, if the team doesn't want to continue using XP, stop.

If only one or two people refuse to use XP and they're interested in working on another project, let them transfer so the rest of the team can use XP. If no such project is available, or if a significant portion of the team is against using XP, don't use it.

Prerequisite #3: A Colocated Team

XP relies on fast, high-bandwidth communication for many of its practices. In order to achieve that communication, your team needs to sit together in the same room.

If Your Team Isn't Colocated...

Colocation makes a big difference in team effectiveness. Don't assume that your team can't sit together; be sure to include the possibility of bringing the team together is your first option.

With that said, it's okay if one or two noncentral team members are off-site some of the time. You'll be surprised, though, at how much more difficult it is to interact with them. (Actually, they're no more difficult to interact with than before; it's the rest of the team that's improved.) Talk with your mentor about how to best deal with the problem.

If a lot of people are off-site, if a central figure is often absent, or if your team is split across multiple locations, you need help beyond this book. You can use XP or another agile method with a distributed team, but it's a complicated problem that's outside of the scope of our discussion. Ask your mentor for help and see Sit Together in Chapter 6 for more ideas.

Prerequisite #4: On-Site Customers

The on-site customers' decisions determine the value of the software.

On-site customers are critical to the success of an XP team. They, led by the product manager, determine which features the team will develop. In other words, their decisions determine the value of the software.

Of the on-site customers, the product manager is likely the most important. She makes the final determination of value. A good product manager will choose features that provide value to your organization. A poor product manager will dither time away on inconsequential features.

Domain experts, and possibly interaction designers, are also important. They take the place of an upfront requirements phase, sitting with the team to plan upcoming features and answering questions about what the software needs to do.

If Your Product Manager is Too Busy to Be On-Site...

If you have an experienced product manager who makes high-level decisions about features and priorities, but who isn't available to sit with the team full-time, you may be able to ask a business analyst or one of the other on-site customers to act as a proxy. The proxy's job is to act in the product manager's stead to make decisions about details while following the actual product manager's high-level decisions.

This can work well if your proxy has the authority to act in place of the product manager. If the proxy is unable to answer questions on his own and needs to confirm every decision with the real product manager, he will introduce too many delays for this book's approach to XP to work well.

If Your Product Manager is Inexperienced...

This may be okay as long as she has a more experienced colleague she turns to for advice.

If You Can't Get a Product Manager At All...

Although good product managers are in high demand, the absence of a product manager is a big danger sign. The right person for the job may not have the title of "product manager" (see Real Customer Involvement in Chapter 6), but XP requires that somebody with business expertise take responsibility for determining and prioritizing features.

Remind your organization of the cost of development (presumably, hundreds of thousands of dollars) and the value the software will bring to them (hopefully, millions of dollars). That value hinges on the participation of a good product manager. Is that really something they want to scrimp on?

If you can't find a product manager, someone from the development team can play the part. However, this may be a dangerous approach, because this person is unlikely to have the business expertise to deliver an organizational success. If you can't get a product manager, talk with your mentor about how to compensate.

If You Can't Get Other On-Site Customers...

Because XP doesn't have an up-front requirements phase, the work of figuring out requirements happens concurrently with software development. This compresses the overall schedule, but it means that at least one person—and usually several—needs to work on requirements full-time.

Unless you have a small team, this work is probably more than a product manager can handle alone. Typically, the product manager delegates the details to a set of domain experts. In applications that involve a sophisticated user interface, a interaction designer may be involved as well. This allows the product manager to focus on coordinating with stakeholders and resolving questions of value and priorities.

Some business analysts may be domain experts. Be careful of using business analysts that aren't already experts in the domain; although they can relay information from true experts, this process invariably introduces misunderstandings and delays.

As long as somebody is playing the on-site customer role, you can use XP. However, the less expertise your on-site customers have, the more risk there is to the value of your software.

Prerequisite #5: The Right Team Size

I wrote this book for teams as large as 20 people and as small as one person. For teams new to XP, however, I recommend four to six programmers and no more than 12 people on the team (see The XP Team in Chapter 3). I also recommend having an even number of programmers so that everyone can pair program (see Pair Programming in Chapter 5). If you have ongoing support needs, add one more programmer for a total of five or seven so the team can have a batman (see Iteration Planning in Chapter 8).

Teams with fewer than four programmers are less likely to have the intellectual diversity they need. They'll also have trouble using pair programming, an important support mechanism in XP. Large teams introduce coordination challenges. Although experienced teams will handle those challenges smoothly, a new XP team will struggle.

If You Don't Have Even Pairs...

The easiest solution is to add or drop one programmer so you have even pairs. If you can't do that, the XP practices are still appropriate for you, but try to find useful nonproduction code work for the programmer who isn't pairing. This will help the team consistently apply XP's technical practices and will improve code quality.

If Your Team is Larger than Seven Programmers...

The coordination challenges of a large team can make learning XP more difficult. Consider hiring an experienced XP coach to lead the team through the transition. You may also benefit from hiring another experienced XP programmer to assist the coach in mentoring the team.

If your team is larger than ten programmers, you need guidance that's outside of the scope of this book. Hire a coach with experience in scaling XP to large teams.

If Your Team is Smaller Than Four Programmers...

Most of the XP practices are still appropriate for you, but you probably won't be able to pair program much. In this situation, it's best if your team members are conscientious programmers who are passionate about producing high-quality code. That passion will help them apply the technical practices with discipline.

You may have trouble getting on-site customers to sit with you full-time. Instead, sit close to them so that you can get their attention when you need it.

If You Have Many Developers Working Solo...

Some organizations—particularly IT organizations—have a lot of small projects rather than one big project. They structure their work to assign one programmer to each project.

Although this approach has the advantage of connecting programmers directly with projects, it has several disadvantages. It's high risk: every project is the responsibility of one programmer, so that any programmer who leaves orphans a project. Her replacement may have to learn it from first principles.

Code quality can also be a challenge. Projects don't benefit from peer review, so the code is often idiosyncratic. Stovepipe systems, in which each programmer solves the same problem in different ways, appear. Junior programmers, lacking the guidance of their more senior peers, create convoluted, kludgey systems and have few opportunities to learn better approaches. Senior programmers, not realizing the inexperience of their more junior peers, create overly sophisticated code that others have trouble understanding.

You may be able to combine four to seven of these programmers into a single XP team that works on one project at a time, which allows it to complete projects more quickly (see Release Planning in Chapter 8). By working together, senior developers have the opportunity to mentor junior developers, and the team can eliminate stovepipe systems.

Combining your programmers into a single team has some drawbacks. The biggest is likely to be a perceived lack of responsiveness. Although projects will be finished more quickly, customers will no longer have a dedicated programmer to talk to about the status of their projects. The team will only work on one project at a time, so other customers may feel they are being ignored.

To resolve these problems, consider dedicating one programmer to deal with customer requests and minor changes (see Iteration Planning in Chapter 8). You'll also need an influential, unbiased business person to play the product manager role, addressing conflicts between customers and making prioritization decisions.

Prerequisite #6: Use All the Practices

You may be tempted to ignore or remove some practices, particularly the ones that make team members uncomfortable. Be careful of this. XP is designed to have very little waste. Nearly every practice directly contributes to the production of valuable software.

For example, pair programming supports collective code ownership, which is necessary for refactoring. Refactoring allows incremental design and architecture. Incremental design and architecture enables customer-driven planning and frequent releases, which are the key to XP's ability to increase value and deliver successful software.

XP doesn't require perfection—it's okay if you accidentally misapply a practice from time to time—but it rarely works well if you arbitrarily remove pieces.

If Practices Don't Fit...

You may feel that some XP practices aren't appropriate for your organization. That may be true, but it's possible you just feel uncomfortable or unfamiliar with a practice. Are you sure that practice won't work, or do you just not want to do it? XP will work much better for you if you give all of the practices a fair chance rather than picking and choosing the ones you like.

If you're sure a practice won't work, you need to replace it. For example, in order to achieve the benefits of collective code ownership without pair programming, you must provide another way for people to share knowledge about the codebase. (You'll also have to find ways to replace the other benefits of pairing.)

Replacing practices requires continuous refinement and an in-depth understanding of XP. Ask your mentor for help and consider hiring an experienced XP coach.

Recommendation #1: A Brand-New Codebase

Easily-changed code is vital to XP. If your code is cumbersome to change, you'll have difficulty with XP's technical practices, and that difficulty will spill over into XP's planning practices.

XP teams put a lot of effort into keeping their code clean and easy to change. If you have a brand-new codebase, this is easy to do. If you have to work with existing code, you can still practice XP, but it will be more difficult. Even well-maintained code is unlikely to have the simple design and suite of automated unit tests that XP requires (and produces). New XP teams often experience an epiphany between the second and fourth months. "This is the best code I've ever worked with!" they say, and start to see the power of XP.

To understand and appreciate XP's technical practices fully, you need to experience the practices meshing together to give you complete confidence in your code, tests, and build. You need to feel the delight of making big improvements with small changes. You're unlikely to have that experience when working with existing code. If you can, save preexisting code for experienced XP teams.

If You Have Preexisting Code...

You can dig your way out of this hole. See "Applying XP to an Existing Project," later in this chapter.

Recommendation #2: Strong Design Skills

Simple, easily-changed design is XP's core enabler. This means at least one person on the team—preferably a natural leader—needs to have strong design skills.

It's hard to tell if somebody has strong design skills unless you have strong design skills yourself. One clue to look for is an understanding and appreciation of domain-driven design. It requires a crucial shift in thinking—from imperative procedural design to declarative object-oriented design—that programmers with poor design skills can have difficulty grasping.

If No One Has Strong Design Skills...

You'll probably do as well with XP as you would be with any method—perhaps more, because XP includes specific technology practices and advice. However, that doesn't mean you'll be successful. Take it slow and steady, and seek out as much experienced help as you can get.

Meanwhile, start learning! [Evans]' Domain-Driven Design is a good place to start, as is [Fowler 2002a]'s Patterns of Enterprise Application Architecture. Consider taking a course or hiring somebody to join the team as a mentor. Be careful, though—strong design skills, while essential, are surprisingly rare. Ask someone with good design skills to help you vet your choice.

Recommendation #3: A Language That's Easy to Refactor

XP relies on refactoring to continuously improve existing designs, so any language that makes refactoring difficult will make XP difficult. Of the currently popular languages, object-oriented and dynamic languages with garbage collection are the easiest to refactor. C and C++, for example, are more difficult to refactor.

If Your Language Is Hard To Refactor...

You can still use XP, but be sure to include someone on your team who has experience with refactoring in your language, if you can.

Recommendation #4: An Experienced Programmer-Coach

Some people are natural leaders. They're decisive, but appreciate others' views; competent, but respectful of others' abilities. Team members respect and trust them. You can recognize a leader by her influence—regardless of her title, people turn to a leader for advice.

Leadership is independent of title or position. You can identify leaders by their followers, not by their desire to give orders. To identify the real leaders on your team, look for the people that team members want to follow.

XP relies on self-organizing teams. Such a team doesn't have a predefined hierarchy; instead, the team decides for itself who is in charge of what. These roles are usually informal. In fact, in a mature XP team, there is no one leader. Team members seamlessly defer leadership responsibilities from one person to the next, moment to moment, depending on the task at hand and the expertise of those involved.

When your team first forms, though, it won't work together so easily. Somebody will need to help the team remember to follow the XP practices consistently and rigorously. This is particularly important for programmers, who have the most difficult practices to learn.

In other words, your team needs a coach. The best coaches are natural leaders—people who remind others to do the right thing by virtue of who they are rather than the orders they give. Your coach also needs to be an experienced programmer so she can help the team with XP's technical practices.

If You Have No Obvious Coach...

Explain the situation to the team and ask them to choose a coach by consensus. In other words, ask them to pick one person that they can all agree would be a good coach.

In consensus decisions, everyone has a veto. A quick way to perform a consensus vote is to ask everyone to hold their thumbs out. Thumbs up means "I agree". Thumbs sideways means "I'll go with the team's decision." Thumbs down means "I disagree and want to explain why."

If you can't pick a coach by consensus, your team may be too fractured to use XP. If there's someone you can hire that the team would trust, that may help. Be sure to tell whoever you hire that you weren't able to reach consensus on this issue—an experienced XP coach will see it as a danger sign and should speak to team members before accepting.

If Your Leaders are Inexperienced...

Good leaders aren't always experienced developers, but a good coach should look for subtle cues that indicate upcoming problems, which does require experience. An experienced developer is your best coach.

If your leaders are inexperienced, you may want to try pair coaching. Pick one person who's a good leader and one person who has a lot of experience. Make sure they get along well. Ask the two coaches to work together to help the team remember to practice XP consistently and rigorously.

If You're Assigned a Poor Coach...

Your organization may assign somebody to be coach who isn't a good leader. In this case, if the assigned coach recognizes the problem, pair coaching may work for you.

If the assigned coach doesn't recognize the problem and he's damaging the team's ability to function, discuss the situation with your mentor or a manager you trust. This is a delicate situation that requires context-specific advice.

Recommendation #5: A Friendly and Cohesive Team

XP requires that everybody work together to meet team goals. There's no provision for someone to work in isolation, so it's best if team members enjoy working together.

If Your Team Doesn't Get Along...

XP requires people to work together. Combined with the pressure of weekly deliveries, this can help team members learn to trust and respect each other. However, it's possible for a team to implode from the pressure. Try including a team member who is level-headed and has a calming influence.

If team members won't even attempt to work together, don't use XP. If there's just one person whose behavior encourages other people's bad behavior, you might be able to solve the problem by moving him to a different team.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.