AoAD2 Practice: Retrospectives

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Retrospectives

Audience
Whole Team

We continually improve our work habits.

No process is perfect. As “Key Idea: Continuous Improvement” on p.XX describes, your team should constantly update and improve your development process. Retrospectives are a great tool for doing so.

Types of Retrospectives

The most common retrospective, the heartbeat retrospective, occurs on a regular cadence. (It’s also known as an iteration retrospective.) For teams using iterations, it occurs at the end of every iteration. For teams using continuous flow, it occurs at a preset time every week or two.

In addition to heartbeat retrospectives, you can also conduct longer, more intensive retrospectives at crucial milestones. These milestone retrospectives give you a chance to reflect more deeply on your experiences and condense key lessons to share with the rest of your organization.

Milestone retrospectives are out of the scope of this book. They work best when conducted by neutral third parties, so consider bringing in an experienced retrospective facilitator. Larger organizations may have such facilitators on staff (start by asking the HR department), or you can bring in an outside consultant. If you’d like to conduct them yourself, [Derby and Larsen 2006] and [Kerth 2001] are great resources.

How to Conduct a Heartbeat Retrospective

The whole team should participate in each retrospective. So people feel safe to speak their minds freely, people outside the team should not attend, with the possible exception of the facilitator.

Anybody on the team can facilitate. In fact, it’s best to switch facilitators frequently. That will help keep it interesting. Start with people who have facilitation experience. Once the retrospective is running smoothly, give the rest of the team a chance to facilitate.

The facilitator does not otherwise participate in the retrospective.

The facilitator does not otherwise participate in the retrospective. Their role is to keep the retrospective on track and to ensure that everyone’s voice is heard. If your team has trouble staying neutral, teams can trade facilitators, so that each team has a neutral, outside facilitator. Be sure that each facilitator agrees to keep everything they happens during the retrospective confidential.

I timebox my retrospectives to exactly one hour. Your first few retrospectives will probably run long. Give it an extra half-hour, but don’t be shy about politely wrapping up and moving to the next step. The whole team will get better with practice, and the next retrospective is only a week or two away.

As [Derby and Larsen 2006] describes, a retrospective consists of five parts: Set the Stage; Gather Data; Generate Insights; Decide What to Do; and Close the Retrospective. Below, I describe a simple, effective approach. Don’t try to match the timings exactly; let events follow their natural pace.

After you’ve acclimated to this format, change it up. The retrospective is a great venue for trying new ideas. See “Retrospectives Alternatives and Experiments” on p.XX for suggestions.

Allies
Safety
Team Dynamics

A word of caution before you begin: Retrospectives can be damaging when used to attack each other. If your team is having trouble treating each other with respect, focus on safety and team dynamics first.

Step 1: The Prime Directive (5 minutes)

In his essay, “The Effective Post-Fire Critique,” New York City Fire Department Chief Frank Montagna writes:

Firefighters, as all humans, make mistakes. When firefighters make a mistake on the job, however, it can be life-threatening to themselves, to their coworkers, and to the public they serve. Nonetheless, firefighters will continue to make mistakes and on occasion will repeat a mistake.

Never use a retrospective to place blame or attack individuals.

Everyone makes mistakes, even when lives are on the line. The retrospective is an opportunity to learn and improve, and everybody needs to be safe to share their experiences and opinions. The team should never use the retrospective to place blame or attack individuals.

As facilitator, it’s your job to nip destructive behavior in the bud. To help remind people of the need for psychological safety, I start each retrospective by repeating Norm Kerth’s Prime Directive:

Regardless of what we discover today, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

If you’re in a physical team room, you can write the Prime Directive on a flip chart and bring it to each retrospective. If you have a virtual team room, you can make it part of a virtual whiteboard that’s dedicated to retrospectives.

Ask each attendee in turn if they agree to the Prime Directive and wait for a verbal “yes.” If they don’t agree, ask if they can set aside their skepticism just for this one meeting. If they still won’t agree, postpone the retrospective. There may be an interpersonal issue that needs to be addressed before people can speak with the openness and honesty the retrospective requires. If you’re not sure what the issue is, ask a mentor for help.

Asking for verbal agreement will feel awkward for some participants, but it serves an important purpose. First, if somebody truly objects, they’re more likely to say so if they have to agree out loud. Second, if somebody speaks up once during a retrospective, they’re more likley to speak again. Verbal agreement encourages participation.

Step 2: Brainstorming (20 minutes)

If everyone agrees to the Prime Directive, write the following categories on the whiteboard (or pre-prepare them):

  • Enjoyable

  • Frustrating

  • Puzzling

  • Keep

  • More

  • Less

Ask the team to use simultaneous brainstorming (see “Work Simultaneously” on p.XX) to think about what’s happened since the last retrospective and write down their reactions (the things they found enjoyable, frustrating, or puzzling) and preferences (the things they want to keep, do more of, or do less of). Write one per card and be sure to include the category on the card.

If people have trouble getting started, briefly recap what’s happened since the last retrospective. (”On Wednesday morning, we had our task planning session...”) Pause after each point to give people a chance to write down ideas. Other people can chime in with their recollections as well.

As the ideas wind down, check the time. If you have extra time remaining, let the silence stretch out. Someone will often say something that they have held back, and this may start a new round of ideas. If you’re running out of time, though, you can move on to the next step.

Step 3: Mute Mapping (15 minutes)

Next, use mute mapping to sort the cards into clusters. When they’re done, use dot voting to select the cluster that the team will focus on improving. After the voting ends, one cluster should be the clear winner. If not, don’t spend a lot of time choosing. Flip a coin or something.

Discard the cards from the other clusters. If someone wants to take a card to work on individually, that’s fine, but not necessary. Remember, you’ll do another retrospective in a week or two. Important issues will recur.

Frustrated that your favorite category lost? Wait a few months. If it’s important, it will win eventually.

Step 4: Retrospective Objective (20 minutes)

Come up with experiments that might make things better.

Now it’s time to come up with options for improvement. Ask the team to brainstorm ideas for improving the selected category. This can involve any idea you can think of: performing some action, changing your process, changing a behavior, or something else entirely. Don‘t try to come up with perfect or complete solutions; just come up with experiments that might make things better.

Ally
Circles and Soup

Allow this conversation to go on for several minutes. If people are having trouble thinking of ideas, try asking them “why” questions: Why does the current approach need improvement? Why doesn’t it work as is? Why is it done this way? It can also help to think in terms of circles and soup: what the team controls and what they influence.1

1XXX Revise when final name of “circles and soup” practice is decided.

Don’t go into a lot of detail. A general direction is good enough. For example, if “pairing” was the category selected, then “switch pairs more often,” “ping-pong pairing,” and “switch pairs on a schedule” are all valid ideas.

The group may coalesce around a single good idea. Other times, there might be several competing proposals. If there are, conduct another dot vote to choose one. This is your retrospective objective: the improvement that the whole team will work toward until the next retrospective. Limit yourself to just one, so the team can focus.

Once you have a retrospective objective, ask somebody to volunteer to work out the details and follow through. It won’t be that person’s job to push or own the objective—that’s for the whole team—but they’ll help people remember the objective when needed. Other team members can volunteer to help if they want.

Wrap up the meeting with a consent vote. (See “Seek Consent” on p.XX.) When everybody consents, the retrospective is over. If you can’t reach consent for some reason, choose another idea or start over at the next retrospective.

Follow Through

If nothing changes, the retrospective didn’t work.

It’s all too easy to leave the retrospective and think, ”Well, that’s done until next week.” Make sure you actually follow through on the retrospective objective. If nothing changes, the retrospective didn’t work.

Ally
Informative Workspace
Stand-Up Meetings

To help the team follow through, make the retrospective objective visible. If you decided to do something, add those tasks to your plan. If you’re changing your process, update your planning boards to visualize the change. If you want people to change their behavior, track it with a big visible chart. If you’re changing a working agreement, update your working agreements poster.

Check in on the retrospective objective every day. The stand-up meeting can be a good place to check in and remind team members to follow through.

Questions

Despite my best efforts as facilitator, our retrospectives always degenerate into blaming and arguing. What can I do?

Ally
Team Dynamics
Safety

Hold off on retrospectives, for now, and focus on team dynamics and establishing psychological safety instead.2 If that doesn’t help, you may need outside help. Consider bringing in an organizational development (OD) specialist to help. Your HR department may have someone on staff.

2XXX revist after Team Dynamics and Safety practices are complete.

We come up with good retrospective objectives, but then nothing happens. What are we doing wrong?

Your ideas may be too big. Remember, you only have one week, maybe two, and you have other work to do, too. Try making plans that are smaller scale—perhaps a few hours of work—and follow up every day.

Ally
Slack

Another possibility is that you don’t have enough slack in your schedule. When you have a completely full workload, nonessential tasks such as improving your work habits go undone. (The sad irony is that improving your work habits will give you more time.)

Finally, it’s possible that the team doesn’t feel like they truly have a voice in the retrospective. Take an honest look at the way you conduct the retrospective. Are you leading the team by the nose rather than facilitating? Consider having someone else facilitate the next one.

Some people won’t speak up in the retrospective. How can I encourage them to participate?

It’s possible they’re just shy. It’s not necessary for everyone to participate all the time. Try starting your next retrospective with an icebreaker activity and see if that helps.

On the other hand, they may have something they want to say, but don’t feel safe doing it. In that case, focus on developing psychological safety in the team.

One group of people (such as testers) always gets outvoted in the retrospective. How can we meet their needs, too?

Over time, it’s likely that every major issue will get its fair of attention. Give the retrospective a few months before deciding that a particular group is disenfranchised. One time I worked with had a few testers that felt their priority was being ignored. A month later, after the team had addressed another issue, the testers’ concern was on the top of everyone’s list.

If time doesn’t help, you can use weighted dot voting. Give people with under-represented skills more votes.

Our retrospective always takes too long. How can we go faster?

As a facilitator, it’s okay to be decisive about wrapping things up and moving on. There’s always next time. If the group is taking a long time brainstorming ideas or mute mapping, you might say something like, “Okay, we’re running out of time. Take two minutes to write down your final thoughts (or make final changes) and then we’ll move on.”

That said, I prefer to let a retrospective take its natural course during the first month or so, even if that means running long. This allows people to get used to the flow of the retrospective without stressing too much about timelines.

The retrospective isn’t accomplishing much. Can we do it less often?

If your team is fluent in your chosen fluency zones and everything’s running smoothly, it’s possible that there’s not much left to improve. In that case, you could try conducting retrospectives less frequently, although you should continue to have one at least every month.

That’s usually not the case, though. It’s more likely that the retrospective has just gotten stale. Try changing it up. Switch facilitators and try new activities or focuses.

Prerequisites

Ally
Safety

The biggest danger in a retrospective is that it will become a venue for acrimony rather than for constructive problem solving. Make sure you’ve created an environment where people are able to share their true opinions. Don’t conduct retrospectives if you have team members who tend to lash out, attack, or blame others.

Indicators

When your team conducts retrospectives well:

  • Your ability to develop and deliver software steadily improves.

  • The whole team grows closer and more cohesive.

  • Each specialty within the team gains respect for the issues other specialties face.

  • Team members are honest and open about successes and failures.

  • The team is comfortable with change.

Alternatives and Experiments

Every retrospective format gets stale over time. Change it up! The format in this book is an easy starting point, but as soon as you have it running smoothly, experiment with other ideas. [Derby and Larsen 2006] is a good resource for learning about how retrospectives are constructed and it also has a variety of activities you can try. Once you’ve absorbed its ideas, see https://www.tastycupcakes.org for more ideas.

Some people find that an hour is too constraining to conduct a satisfying retrospective, and prefer 90 minutes, or even two hours. Feel free to experiment with longer and shorter lengths. Some activities, in particular, will need more time. As you experiment, conduct a brief “retrospective on the retrospective” to evaluate which retrospective experiments should be repeated and which shouldn’t. Chapter 8 of [Derby and Larsen 2006], “Activities to Close the Retrospective,” has ideas.

In addition to trying new activities, you can also experiment with completely different approaches to process improvement. Arlo Belshee, for example, tried a continuous approach, where people put observations in a jar throughout the week, then reviewed those observations at the end of the week. Woody Zuill has an exercise he calls “turn up the good:” at the end of every day, conduct a five-minute retrospective to choose something that went well and decide how to do it even more.

Get familiar with normal heartbeat retrospectives first, though, so you can tell if your experiments are an improvement or not.

Further Reading

Project Retrospectives [Kerth 2001] is the definitive resource for milestone retrospectives.

Agile Retrospectives: Making Good Teams Great [Derby and Larsen 2006] picks up where Kerth leaves off, discussing techniques for conducting all sorts of Agile retrospectives.

“The Effective Post-Fire Critique” [Montagna 1996] is a fascinating look at how a life-and-death profession approaches retrospectives.

XXX Retrospectives Antipatterns? (Aino Corry)

XXX Thomas Owens recommends:

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.