• Next: Assess Your Agility
• Previous: Is XP Right For Us?
• Up: Chapter 4: Adopting XP

in 99 words

Any big change is a challenge. When adopting XP, expect the first few months to be chaotic, and a little painful, as everyone gets up to speed. Give yourself four to nine months to feel truly comfortable with your new process.

If you can, adopt XP all at once. This works best when you have a brand-new codebase. It's the easiest way to learn.

When applying XP to an existing project, take an incremental approach. Start by introducing the structural practices. Follow up with the technical practices, paying down technical debt and chipping away at your bug backlog.

as haiku

freshly turned soil--
fertile, waiting, it smells like
possibility

Commentary

Marick's Missing Manifesto

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

Go!

Are you ready to adopt XP? Great! Your first step is to arrange for your open workspace (see Sit Together in Chapter 6). Start solving this problem now. It will probably take longer than you expect.

Next, find an appropriate project for the team to work on. Look for a project that's valuable, but be wary of projects that will be under intense scrutiny. You need room to make mistakes as you learn.

Avoid taking a project with low value as a "learning opportunity." You'll have trouble involving customers and achieving an organizational success. Your organization could view the project as a failure even if it's a technical success.

At the same time, figure out who will be on your team. The XP Team in Chapter 3 provides some suggestions for team structure. Talk with your project's executive sponsor and other stakeholders about who to include as your on-site customers. (See Real Customer Involvement in Chapter 6 for ideas.) Be sure your team members want to try XP.

As you're forming your team, consider hiring an experienced XP coach to work with the team full-time. Although an outside coach isn't necessary—I learned XP by reading about it and trying it—a good coach will make things go more smoothly.

Extreme Shopping

As your project start date draws near, you'll need supplies for the team's open workspace. The following is a good shopping list:

Equipment:

• Pairing stations (see Pair Programming in Chapter 5).

• A dedicated build machine (see Continuous Integration in Chapter 7).

• Noise dampening partitions to define your team's workspace and prevent noise pollution (see Sit Together in Chapter 6).

• Plenty of wall-mounted whiteboards for discussions and charts (see Informative Workspace in Chapter 5). Ferrous ("magnetic") whiteboards are best because you can stick index cards to them with magnets.

• Two big magnetic whiteboards for your release and iteration plans. I like using a two-sided, six-foot magnetic whiteboard on wheels—it allows me to move the plan into a meeting room. Some teams prefer corkboards, but I think they make it too hard to slide cards around.

• A large plastic perpetual calendar (three months or more) for marking important dates and planned absences (see Informative Workspace in Chapter 5).

• A plush toy to act as your integration token (see Continuous Integration in Chapter 7).

• Miscellaneous toys and conversation pieces to inspire discussion and interaction among team members.

• Any other equipment you normally use.

Software:

• A unit-testing tool such as the xUnit family (see Test-Driven Development in Chapter 9).

• An automated build tool such as the Ant family (see Ten-Minute Build in Chapter 7).

• Any other software you normally use.

Continuous integration software is not required (see Continuous Integration in Chapter 7).

Supplies:

• Index cards—start with 5,000 of white and 2,000 of each color you want. Be sure to choose colors that all members of your team can distinguish (seven to ten percent of men have a degree of color-blindness).

• Pencils for index cards. (Don't use pens; you'll need to make changes from time to time.)

• Food (see Energized Work in Chapter 3 and Trust in Chapter 6).

• Flip charts and something sticky but non-marking (such as blue painters' tape or poster tack) for hanging charts (see Informative Workspace in Chapter 5).

• Dry-erase markers for whiteboards, water-based flip-chart markers for flip charts,¹ and wet-erase markers for the perpetual calendar. (Be sure the markers are easily distinguishable!)

• Magnets for sticking papers to whiteboards. Office supply stores sell thin, flexible magnets that work well. Make sure they're powerful enough to hold an index card or two to the board.

• Any other supplies you normally use.

¹Avoid permanent markers; they bleed through the paper and will damage your whiteboard if you use the wrong pen by mistake.

The Challenge of Change

It's a fact of life: change makes people uncomfortable. XP is probably a big change for your team. If you previously used a rigid, document-centric process, XP will seem loose and informal. If you previously had no process, XP will seem strict and disciplined. Either way, expect team members to be uncomfortable. This discomfort can extend into the larger organization.

Discomfort and a feeling of chaos is normal for any team undergoing change, but that doesn't make it less challenging. Expect the chaotic feeling to continue for at least two months. Give yourselves four to nine months to feel truly comfortable with your new process. If you're adopting XP incrementally, it will take longer.

To survive the transformation, you need to know why you are making this change. What benefits does it provide to the organization? To the team? Most importantly, what benefits does it provide to each individual? As you struggle with the chaos of change, remember these benefits.

A supportive work environment is also important. Team members are likely to experience defense reactions to the lack of familiar structure. Expect mood swings and erratic behavior. Some team members may lash out or refuse to cooperate. Acknowledge the discomfort people are experiencing, and help team members find constructive outlets for their frustration.

The Satir Change Model is one way of understanding teams' reactions to change. [Smith] has a good article on the Satir model at http://www.stevenmsmith.com/my-articles/article/the-satir-change-model.html that includes tips for helping team members through each stage.

Your stakeholders may be uncomfortable with your new approach to planning and reporting progress. Managers and executives may see the team's initial chaos as a sign that XP won't work for your team. To help everyone feel more comfortable, consider giving them this pledge:

Our pledge to users, management, and other stakeholders.

We promise to:

• Make steady progress

• Finish the features that you consider most valuable first

• Show you working software that reflects our progess every week, on (day of week) at (time) in (location)

• Be honest and open with you about our successes, challenges, and what we can reasonably provide

In return, we ask you to be patient with changes in our productivity and understanding of our mistakes as we learn this new way of working over the next two quarters.

Final Preparation

Before starting XP, it's a good idea to discuss working agreements—that is, which practices your team will follow and how your practice of XP will differ from what I describe in this book. (I recommend following the book as closely as you can until you've had several months of experience.) Discuss your roles and what you expect from each other. It's best to hold these conversations as a collaborative team discussions. Try to avoid assigning roles or giving people orders.

In the final weeks before starting your new XP project, review the practices in Part II. Try some of the practices in your current work and consider taking courses on the practices that seem challenging.

When you've finished these preparations, if you have a greenfield project—meaning your team is creating a new codebase from scratch—you're ready to go. Review the practices in Part II one more time, take a deep breath, and start your first iteration.

"Wait!" you may wonder. "Isn't there a way we can ease into this?"

Well... yes. You can follow the incremental approach that legacy projects use, but if you have a greenfield project, it's actually easier and faster to adopt all the practices at once. It's the chaos and uncertainty of change that makes adopting XP difficult, not the practices themselves. If you adopt XP incrementally, every new practice will disrupt the equilabrium you'll be fighting to achieve. You'll actually extend the period of chaos and uncertainty, making the transition all the more difficult. In my experience, teams that adopt XP incrementally make substantial improvements, but it's the team that adopt it all at once that really excel.

Be bold. You have the right people, the right workplace, and the will to succeed. Do it!

Second Adopter Syndrome

I've noticed a suprising trend among companies that adopt XP: the first team is often very successful, inspiring the organization to use XP on more projects, but then this second wave of XP projects struggles.

I call this second adopter syndrome. My theory is that the first XP project gets all of the support it needs: eager participants, organizational patience, outside help, and a valuable but noncritical project.

Then, thinking that employees now understand XP, the organization provides little support for the second wave of projects. They staff the teams with people who don't want to use XP, provide no outside help, and impose more schedule pressure.

To avoid second adopter syndrome, remember that success on one team doesn't automatically guarantee success on another team. Every team needs support when it adopt XP for the first time.

Applying XP to a Brand-New Project (Recommended)

When starting a brand-new XP project, expect the first three or four weeks to be pretty chaotic as everyone gets up to speed. During the first month, on-site customers will be working out the release plan, programmers will be establishing their technical infrastructure, and everyone will be learning how to work together.

Plan and build infrastructure incrementally throughout the entire project.

Some people think the best way to overcome this chaos is to take a week or two at the beginning of the project to work on planning and technical infrastructure before starting the first iteration. Although there's some merit to this idea, an XP team should plan and build technical infrastructure incrementally and continuously throughout the project as needed. Starting with a real iteration on the first day helps establish this good habit.

Ally

Release Planning

Your very first activity will be to plan your first iteration. Normally, this involves selecting stories from the release plan, but you won't have a release plan yet. Instead, think of one feature that will definitely be part of your first release. Brainstorm a few must-have stories for that feature. These first few stories should sketch out a "vertical slice" (see Figure 8-3) of your application. If the application involves user interaction, create a story to display the initial screen or web page. If it includes reporting, create a story for a bare-bones report. If it requires installation, create a story for a bare-bones installer.

Don't expect much from these initial stories. The programmers' estimates for them will be fairly high because they need to establish some technical infrastructure. As a result, the stories should do very little. The report might display headers and footers, but no line items. The installer might just be a .zip file. The initial screen could have nothing more than your logo on it.

Ally

Estimating

These basic stories will give you ideas for more stories that will add missing details. Brainstorm ten to twenty in the first planning session and have the programmers estimate them. These should keep the programmers busy for several iterations. Try to choose stories that the programmers already understand well; this will reduce the amount of time customers need to spend answering programmer questions so they can focus on creating the release plan.

Ally

Iteration Planning

Iteration planning is a little more difficult during the first iteration because you haven't established a velocity yet. Just make your best guess about what your velocity might be. (Some teams add up the available programmer-days and divide by π.) During the iteration, work on just one or two stories at a time and check your progress every day. This will help you deliver completed stories even if your initial plan is wildly inaccurate.

After you've finished planning, programmers should start establishing their technical infrastructure. Set up an integration machine, create your version control repository, and so forth. (I recommend creating engineering tasks for these items during iteration planning. See Iteration Planning in Chapter 8 for more about the role of engineering tasks in iteration planning.) Once that's set up, start working on your stories.

During the first iteration, it's a good idea to have all the programmers work on the first few stories as a group. Set up a projector so the whole team navigates while one person drives. (See Pair Programming in Chapter 5 for an explanation of driving and navigating.) Sometimes individual programmers (or pairs) peel off to take care of some necessary issue, such as installing a version control system or setting up the programmers' workstations, but for the most part you should work as a team. This reduces the chaos that occurs when multiple people work on a tiny project and allows you to jointly establish initial conventions, such as project structure, filenames and namespaces, and basic design choices.

Ally

Coding Standards

After the first few days, the fundamentals should be well-established and the project should be large enough for people to work on separate parts without unduly interfering with each other. At this point, you can break into pairs and work normally. It's also a good time to schedule your first coding standards discussion. For that first meeting, you can usually just document what you agreed on while working as a group.

Allies

Vision

Release Planning

While the programmers are working on stories, customers and testers should work on the vision and release plan. First, work with stakeholders to create the product vision. You probably already have an idea what the vision for the project is; now formalize it. Finalizing the vision could take a few weeks, so while that's in progress, brainstorm the stories for your first feature. Start thinking about other features that you want to include and pick a date for your first release. Decide on your planning horizons as well. (See Release Planning in Chapter 8 for more about planning horizons.)

Each subsequent iteration will be a little easier to plan. The programmers' estimates will stabilize and your velocity will become predictable. You'll be able to estimate the scope of your next release and fill out your planning horizons. The feeling of chaos will begin to subside as the team works in a steady, predictable rhythm.

Applying XP to an Existing Project

Greenfield projects can adopt all the XP practices at once. You'll experience some bumps along the way, but you'll typically have things figured out in four to nine months.

If you're working with an existing codebase that has no tests, particularly one that's been around for a year or more—in other words, if you have a legacy project—you can achieve the same results, but it will take more time. In this case, adopt XP incrementally.

In this discussion, I assume you have a project burdened with a lot of technical debt and a high bug rate. If your situation isn't that bad, this process is still appropriate, if much easier.

The Big Decision

Other than change itself, the biggest challenge in applying XP to an existing project is not writing tests, refactoring, or cleaning up your bug database. The biggest challenge is setting aside enough time to pay down debt.

If you have a typical legacy project, your current velocity is a polite fiction based on shortcuts. In other words, you incur new technical debt in order to meet your deadlines. To improve productivity and reduce bug production, not only do you need to stop incurring new technical debt, you need to set aside extra slack (see Slack in Chapter 8) for paying down the existing technical debt. This double hit will cause your velocity to go down. It might go down a lot.

Fortunately, as your technical debt decreases, your velocity will rise again. Eventually it will surpass your current velocity. This can take a while. Depending on the amount of technical debt you have and how much slack you set aside for paying it down, expect your velocity to remain low for at least a quarter, probably more.

The more slack you provide for paying down technical debt, the lower your velocity will be, but the less time it will take for your velocity to rise again. Think of velocity as cash flow: the more principal you pay on your debt, the less cash you have each week, but the more quickly you can stop paying interest.

Setting aside slack is a painful decision. However, if you don't stop accumulating technical debt, your velocity will continue to decrease and your defect production rate will increase. Eventually, the cost of development will exceed the value of even simple changes. Your organization will either shelve the product or rewrite it at great expense.

Product managers, avoid this fate by acting decisively now. This is your best option for turning a debt-ridden legacy project into a long-term asset.

You can also rewrite the project from scratch or stop for several weeks to do nothing but pay down technical debt. Although these approaches take less effort than incremental debt paydown, they're risky. These efforts often take much longer than expected, and you lose feedback from stakeholders as well as the opportunity to take advantage of new business opportunities in the meantime.

Bring Order to Chaos

The first thing you need to do is bring structure to your project. Many legacy projects have a chaotic approach to planning, even if they started out well.

Start by introducing XP's structural practices. Move the team, including customers and testers, into a shared workspace, start pair-programming, conduct iteration planning and retrospectives, and so forth. Apply:

• All the Thinking practices (Chapter 5).

• All the Collaborating practices (Chapter 6).

• All the Planning practices (Chapter 8).

• Version Control), Collective Code Ownership, and Done Done (described in Chapter 7), and customer reviews (see Incremental Requirements in Chapter 9).

Two-, three-, or even four-week iterations may be best for you. Start with two-week iterations. In particularly challenging environments, you may have trouble making your stories both small and customer-valued (see Stories in Chpater 8). Consider increasing your iteration length, but talk to your mentor (see "Find a Mentor" in Chapter 2) before doing so.

At this point, your method is very similar to the Scrum method. You may find Scrum courses and reading material useful.

Other than working more closely together, the biggest changes will be to planning. Take your existing project plan and convert each line item into a story card. If the stories aren't customer-centric, that's okay for now; once the team is used to working in iterations, the customers and the project manager should start revising the stories to make them more customer-centric.

When you are comfortable with the structural practices, begin introducing technical practices.

Pay Down Technical Debt

Allies

Ten-Minute Build

Continuous Integration

Test-Driven Development

The biggest problem facing legacy projects is usually excessive technical debt. You need to stop the bleeding by preventing more technical debt from occurring. First, create a ten-minute build. Follow up with continuous integration. Introduce test-driven development.

Meanwhile, reduce existing technical debt by introducing extra slack into your iterations (see Slack in Chapter 8). Use it to pay down technical debt as described in that section. At first, your clean-up efforts will seem fruitless, but over time, you'll see greater and greater benefits to quality and productivity. As your code quality improves, introduce the remaining practices in Releasing and Developing.

These first steps will allow you to steadily pay down technical debt while continuing to make progress on new stories. As the bug rate for new code drops, you can start organizing your bug backlog.

Organize Your Backlog

If your team is like most teams, your bug database is full of to-dos, questions, feature requests, and genuine defects. Customers and testers, go through the database and eliminate duplicates and unimportant issues. Close feature requests by turning them into stories or rejecting them. Find another way to address to-dos and questions. When you're done, the only items remaining should be genuine defects.

If users expect to ask questions through your bug database, consider leaving the questions in the database. You risk alienating your users by requiring them to use a different forum.²

²Thanks to Erik Petersen for this insight.

Depending on the size of your bug database, you may not be able to do this work in a single session. Chip away at it every iteration, just as the programmers do with technical debt.

If your bug database is in use by stakeholders, support personnel, or other people outside of the team, find a way to keep new entries clean. You may be able to institute new policies for using the database, but your best approach is probably to review, clean up, and categorize new entries every day.

Fix Important Bugs

Either way, as your bug database becomes a reliable bug repository, make a fix or don't fix decision for each bug. You should probably involve the product manager at some level and you may need the programmers to estimate the cost of fixing some of the bugs.

Close or defer all of the bugs that you decide not to fix in this release. You can revisit them when you plan the next release. At this point, all that remains in the database is bugs that you will fix. Turn these bugs into stories, have the programmers estimate any that remain unestimated, and put them in the release plan.

If you have a lot of bugs, consider spreading bug fixes throughout your plan. Although normally it's better to fix bugs immediately, spreading out the bugs will allow you to deliver feature enhancements in each iteration, which may be better for stakeholder relations.

Over the remainder of the release, fix the bugs and work on preventing their causes as described in No Bugs in Chapter 7. Continue to pay down technical debt and start applying a bit of root-cause analysis as well.

Move Testers Forward

When you start this process, your testers will probably spend their time testing each release prior to delivery. A large part of their workload is likely to be manual regression testing. The programmers' focus on test-driven development will slowly create an automated regression suite and reduce the pressure on the testers.

As time passes, productivity improves, and programmers have less need to pay down technical debt, use your iteration slack to automate the remaining manual regression tests. You may need to create end-to-end tests at first. Over time, refactor the end-to-end tests into more focused unit and integration tests.

With the regression testing burden eliminated and the team producing few new bugs, the testers will have time available for other work. Take advantage of this opportunity to finish integrating the testers into the team. Move them forward in the process so that, rather than testing after a development phase, they help the team produce higher quality code from the beginning. Have them work with customers to find holes in requirements (see Customer Tests in Chapter 9) and begin conducting exploratory testing (see Exploratory Testing in Chapter 9).

Emerge From the Darkness

This process will allow you to reduce technical debt, increase code quality, and remove defects. As you do, productivity will increase. At first, your progress will be imperceptible. Depending on the amount of technical debt you face, it could take many months to get to the ideal of nearly zero new bugs each month. It will take months more to finish your regression test suite, eliminate the need for a separate pre-release testing phase, and integrate your testers.

As long as each iteration has less debt than the previous, however, you will get there. It will take time and hard work but it will be well worth it. After the first few months, you should start seeing progress in the form of more reliable estimates and more enjoyable programming.

If you don't see progress within two months, there may be something wrong. Talk to your mentor for advice.

Applying XP in a Phased-Based Organization

XP assumes that you use iterations, not phases, which makes using XP in a phase-based environment difficult. If your organization uses a phase-based approach to development, you may be able to use the XP development practices (see Releasing and Developing) even if you can't use the other practices.

Your organization may want to try XP within your existing phase-based structure. Your best course of action is to convince your organization to let you try XP's simultaneous phases. If that doesn't work, you may be able to shoehorn XP into a phase-based structure. It's difficult and the exact approach depends on your organization. The following suggestions are a starting point; talk to your mentor for more specific advice.

Mandatory Planning Phase

Your organization may have a planning phase or planning gate which expects you to deliver a detailed plan. If you can, allocate a month for the planning phase and use it to run four actual iterations. (You may be able to combine the planning phase and analysis phase to get more time.) Use the approach described in Release Planning in Chapter 8 to create your release plan during those first iterations. You'll end up with a good plan and you will have finished some actual software, too.

If you can't use this approach, whatever approach your organization currently uses for planning will be fine, although it probably won't be as accurate as conducting actual iterations.

Mandatory Analysis Phase

Ally

Stories

If your organization conducts an upfront analysis phase, you may receive a requirements document as a fait accompli. In this case, decompose the requirements document into stories. One starting point is to create a story out of each sentence including the words "must", "shall", or "should".

If instead you need to create your own requirements document, XP doesn't have much to add. Use traditional requirements-gathering techniques in this situation, perhaps using iterations and requirements gathering stories for structure.

Requirements documents can't replace on-site customers.

Requirements documents aren't a replacement for a good product manager or on-site customers. Without those people, you will have difficulty filling in missing details in the requirements documents. You will also have more trouble making good schedule/scope trade-offs.

Mandatory Design Phase

Allies

Incremental Design And Architecture

Simple Design

XP assumes the use of incremental design and architecture that is intimately tied to programming with test-driven development. An up-front design phase has little to add to this approach.

If you can, conduct actual XP iterations during the design phase and work on the first stories in your release plan. Use the time to create an initial design and architecture incrementally. Document the results in your design document.

XP focuses on improving and adapting the design throughout the project. Simple design is central to doing so. Dedicated design phases often lead to complex designs, so minimize the amount of time you spend on upfront design if you can.

Mandatory Coding Phase

XP fits well into the coding phase. Break your coding phase into one-week iterations and conduct XP as normal.

Mandatory Testing Phase

XP performs a lot of testing every iteration. A phase-based organization that considers XP to be the coding phase and expects a long testing phase might schedule too little time for coding and too much time for testing. However, testing is an important part of XP and should remain integrated.

Mandatory Deployment Phase

Ally

Ten-Minute Build

With a good build, you should be ready to deploy at the end of any iteration. You can schedule XP's wrap-up activities for the deployment phase.

Extremities: Applying Bits and Pieces of XP

What if your team doesn't meet the conditions for using XP? What then?

Although you won't be able to use all of XP, you may be able to add some XP practices to your existing method. Several practices are easy to adopt and are likely to make an immediate difference:

Iterations

If you struggle with frequent interruptions, try adopting day-long iterations (see Iteration Planning in Chapter 8). Use the planning game (see The Planning Game in Chapter 8) and the team's measured velocity (discussed in Estimating in Chapter 8) to conduct a joint planning session at the beginning of each day, then defer all interruptions until the next planning meeting, which will be less than a day away. Be sure to have programmers estimate their own tasks.

If you aren't interrupted frequently, but still feel a sense of chaos in your planning, try using weekly iterations. In this case, you may also benefit from daily stand-up meetings (see Stand Up Meetings in Chapter 6) and weekly iteration demos (see Iteration Demo in Chapter 6). As time goes on, consider using index cards for planning and a big chart to show upcoming work, as shown in Release Planning in Chapter 8.

Retrospectives

Frequent retrospectives (see Retrospectives in Chapter 5) are an excellent way for your team to adapt and improve its process. If your team has the authority to make any improvements to its process, try scheduling weekly or biweekly retrospectives.

Ten-Minute Build

A fast, automated build will make a big difference to your quality of life, and it will open up opportunities for other improvements as well. See Ten-Minute Build in Chapter 7 for more.

Continuous Integration

Continuous integration not only decreases integration problems, it also drives improvements to your build and tests. See Continuous Integration in Chapter 7 for more.

Test-Driven Development

Although test-driven development (see Test-Driven Development in Chapter 9) isn't as easy to adopt as the other practices, it's very powerful. Test-driven development is the basis for reducing bugs, increasing development speed, improving your ability to refactor, and decreasing technical debt. It can take some time to master, so be patient.

Other Practices

Other XP practices might help, so review Part II. Many of the practices there require the support of other practices, so be sure to read each practice's "Contraindications" section carefully before trying it.

• Next: Assess Your Agility
• Previous: Is XP Right For Us?
• Up: Chapter 4: Adopting XP