The Art of Agile Development: Trust
April 8, 2010
The second edition is now available! The Art of Agile Development has been completely revised and updated with all new material. Visit the Second Edition page for more information, or buy it on Amazon.
- Next: Sit Together
- Previous: Chapter 6: Collaborating
- Up: Chapter 6: Collaborating
in 99 words
For maximum productivity, team members must rely on each other for help. They must take joint responsibility for their work. Trust is essential. To improve trust, foster empathy. Learn about your teammates' joys and concerns. Sitting together helps, as does eating together. Preserve productive teams by keeping them together for multiple projects.
Organizational trust is also essential. Be energetic, deliver on your commitments, and be honest about problems. Show that you care about your stakeholders' goals. Promote your work through friendly openness. Always be honest about your achievements. Avoid the temptation to whitewash problems or misrepresent partially-done work.
Tadpole gazes up:
"Will the sun ever return?"
"Ribbit," replies Frog.
The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.
- Whole Team
We work together effectively and without fear.
When a group of people comes together to work as a team, they go through a series of group dynamics known as "Forming, Storming, Norming, and Performing" [Tuckman]. It takes the team some time to get through each of these stages. They make progress, fall back, bicker, and get along. Over time—often months—and with adequate support and a bit of luck, they get to know each other and work well together. The team jells. Productivity shoots up. They do really amazing work.
What does it take to achieve this level of productivity? The team must take joint responsibility for their work. Team members need to think of the rest of the team as "us", not "them". If a team member notices something that needs doing, she takes responsibility for it, even if it's not her specialty: she does it, finds someone to help her do it, or recruits someone else to take care of it.
Conversely, team members must rely on each other for help. When one member of a team encounters a question that she cannot answer, she doesn't hesitate to ask someone who does know the answer. Sometimes these quick questions turn into longer pairing sessions.
Trust is essential for the team to perform this well. You need to trust that taking time to help others won't make you look unproductive. You need to trust that you'll be treated with respect when you ask for help or disagree with someone.
The organization needs to trust the team, too. XP is strange and different at first. It doesn't provide the normal indicators of progress that managers are accustomed to seeing. It takes trust to believe that the team will deliver a success.
Trust doesn't magically appear—you have to work at it. Here are some strategies for generating trust in your XP team.
Team Strategy #1: Customer-Programmer Empathy
Many organizations I've worked with have struggled with an "us versus them" attitude between customers and programmers. Customers often feel that programmers don't care enough about their needs and deadlines, some of which, if missed, could cost them their jobs. Programmers often feel forced into commitments that they can't meet, hurting their health and relationships.
Sometimes the acrimony is so intense that the groups actually start doing what the others fear: programmers react by inflating estimates and focusing on technical toys at the expense of necessary features; customers react by ignoring programmer estimates and applying schedule pressure. This sometimes happens even when there's no overt face-to-face hostility.
This is a difficult situation with no easy answer. Such badly broken trust takes a long time to heal. Because neither group can force the other to bridge the gap, you have to focus on changing your own attitude.
I find that the biggest missing component in this situation is empathy for the other group's position. Programmers, remember that customers have corporate masters that demand results. Bonuses, career advancement, and even job security depend on successful delivery, and the demands aren't always reasonable. Customers must deliver results anyway.
Customers, remember that ignoring or overriding programmers' professional recommendations about timelines often leads to serious personal consequences for programmers. "Death march teams are the norm, not the exception... [These teams] are often found working 13- to 14-hour days, six days a week... In ["ugly" death march projects], it's not uncommon to see one or two of the project team members collapse from exhaustion, suffer ulcers or a nervous breakdown, or experience a divorce" [Yourdon] (p.ix, p.4, p.61). The commonality of this experience among programmers leads to apathy and cynicism about schedules and commitments.
Sitting together is the most effective way I know to build empathy. Each group gets to see that the others are working just as hard as they are. Retrospectives also help, if your team can avoid placing blame. Programmers can help by being respectful of customer goals and customers can help by being respectful of programmer estimates and technical recommendations. All of this is easier with energized work.
Team Strategy #2: Programmer-Tester Empathy
I've also seen "us versus them" attitudes between programmers and testers, although it isn't quite as prevalent as customer-programmer discord. When it occurs, programmers tend not to show respect for the testers' abilities, and testers see their mission as shooting down the programmers' work.
As with customer-programmer discord, empathy and respect are the key to better relations. Programmers, remember that testing takes skill and careful work, just as programming does. Take advantage of testers' abilities to find mistakes you would never consider, and thank them for helping prevent embarrassing problems from reaching stakeholders and users. Testers, focus on the team's joint goal: releasing a great product. When you find mistakes, it's not an occasion for celebration or gloating. Remember, too, that everybody makes mistakes, and mistakes aren't a sign of incompetence or laziness.
Team Strategy #3: Eat Together
A good way to improve team cohesiveness is to eat together.
Another good way to improve team cohesiveness is to eat together. Something about sharing meals breaks down barriers and fosters team cohesiveness. Try providing a free meal once per week. If you have the meal brought into the office, set a table and serve the food family-style to prevent people from taking food back to their desks. If you go to a restaurant, ask for a single long table rather than separate tables.
Team Strategy #4: Team Continuity
After a project comes to end, the team typically breaks up. All of the wonderful trust and cohesiveness that the team has formed is lost. The next project starts with a brand-new team and they have to struggle through the four phases of team formation all over again.
You can avoid this waste by keeping productive teams together. Most organizations think of people as the basic "resource" in the company. Instead, think of the team as the resource. Rather than assigning people to projects, assign a team to a project. Have people join teams and stick together for multiple projects.
Some teams will be more effective than others. Take advantage of this by using the most effective teams as a training ground for other teams. Rotate junior members into those teams so they can learn from the best, and rotate experienced team members out to lead teams of their own. If you do this gradually, the team culture and trust will remain intact.
Team continuity is an advanced practice—not because it's hard to do, but because it challenges normal organizational structures. While team continuity is valuable, you don't need to do it to be successful.
I know somebody who worked in a company with two project teams. One used XP, met its commitments, and delivered regularly. The team next door struggled: it fell behind schedule and didn't have any working software to show. Yet when the company downsized, it let the XP team members go rather than the other team!
Why? When management looked in on the struggling team, they saw programmers working hard, long hours with their heads down and UML diagrams papering the walls. When they looked in on the XP team, they saw people talking, laughing, and going home at five with nothing but rough sketches and charts on the whiteboards.
Like it or not, our projects don't exist in a vacuum. XP can seem weird and different to an organization that hasn't seen it before. "Are they really working?" outsiders wonder. "It's noisy and confusing. I don't want to work that way. If it succeeds, will they force me to do it too?"
Ironically, the more successful XP is, the more these worries grow. Alistair Cockburn calls them organizational antibodies.1 If left unchecked, organizational antibodies will overcome and dismantle an otherwise successful XP team.
1Via personal communication.
No matter how effective you are at meeting your technical commitments, you're in trouble without the good will of your stakeholders. Yes, meeting schedules and technical expectations helps, but the non-technical, interpersonal skills—soft skills—your team practices may be just as important to building trust in your team.
Does this sound unfair or illogical? Surely your ability to deliver high-quality software is all that really matters.
It is unfair and illogical. It's also the way people think—even programmers. If your stakeholders don't trust you, they won't participate in your team, hurting your ability to deliver valuable software. They might even campaign against you.
Don't wait for them to realize how your work can help them. Show them.
Organizational Strategy #1: Show Some Hustle
"Things may come to those who wait, but only the things left by those who hustle." —Abraham Lincoln2
2Thanks to George Dinwiddie for this quote.
Several years ago, I hired a small local moving company to move my belongings from one apartment to another. When the movers arrived, I was surprised to see them hustle—they moved quickly from the van to the apartment and back. This was particularly unexpected because I was paying them by the hour. There was no need for them to move so quickly.
Those movers impressed me. I felt that they were dedicated to meeting my needs and respecting my pocketbook. If I still lived in that city and I needed to move again, I would hire them in an instant. They earned my goodwill—and my trust.
In the case of a software team, hustle is energized, productive work. It's a sense that the team is putting in a fair day's work for a fair day's pay. Energized work, an informative workspace, appropriate reporting, and iteration demos all help convey this feeling of productivity.
Organizational Strategy #2: Deliver on Commitments
If your stakeholders have worked with software teams before, they probably have plenty of war wounds from slipped schedules, unfixed defects, and wasted money. In addition, they probably don't know much about software development. That puts them in the uncomfortable position of relying on your work, having had poor results before, and being unable to tell if your work is any better.
Meanwhile, your team consumes thousands of dollars every week in salary and support. How do stakeholders know that you're spending their money wisely? How do they know that the team is even competent?
Stakeholders don't know how to evaluate your process, but they can evaluate results. Two kinds of results speak particularly clearly to them: working software and delivering on commitments.
Fortunately, XP teams demonstrate both of these results every week. You make a commitment to deliver working software when you build your iteration and release plans. You demonstrate that you've met the iteration commitment in the iteration demo, exactly one week later, and you demonstrate that you've met the release commitment on your predefined release date.
This week-in, week-out delivery builds trust in stakeholders like nothing I've ever seen. It's extremely powerful. All you have to do is create a plan that you can achieve... and then achieve it.
Organizational Strategy #3: Manage Problems
Did I say "all you have to do?" Silly me. It's not that easy.
First, you need to plan well (see Chapter 8). Second, as the poet said, "The best laid schemes o' mice an' men / Gang aft a-gley."3
3"To a Mouse," by renowned Scottish poet Robert Burns. The poem starts, "Wee, sleekit, cow'rin, tim'rous beastie, O, what a panic's in thy breastie!" This reminds me of how I felt when asked to integrate a project after a year of unintegrated development.
In other words, some iterations don't sail smoothly into port on the last day. What do you do when your best laid plans gang a-gley?
Actually, that's your chance to shine. Anyone can look good when life goes according to plan. Your true character shows when you deal with unexpected problems.
The first thing to do is to limit your exposure to problems. Work on the hardest, most uncertain tasks early in the iteration. You'll find problems sooner, and you'll have more time to fix them.
When you encounter a problem, start by letting the whole team know about it. Bring it up by the next stand-up meeting at the very latest. This gives the entire team a chance to help solve the problem.
If the setback is relatively small, you might be able to absorb it into the iteration by using some of your iteration slack. Options for absorbing the problem include reducing non-critical refactorings, postponing a non-essential meeting, or even (as a team decision) cancelling research time. I personally volunteer to work an hour or so longer each day until we have resolved the problem—or the iteration ends, whichever comes first—when my family commitments permit.
Some problems are too big to absorb no matter how much slack you have. In this case, get together as a whole team as soon as possible and replan. You may need to remove an entire story or you might be able to reduce the scope of some stories. (For more about what to do when things go wrong, see Iteration Planning in Chapter 8.)
The bigger the problem, the sooner you should disclose it.
When you've identified a problem, let the stakeholders know about it. They'll appreciate your professionalism even if they don't like the problem. I usually wait until the iteration demo to explain problems that we solved on our own, but bring bigger problems to stakeholders' attention right away. The product manager is probably the best person to decide who to talk to and when.
The sooner your stakeholders know about a problem (and believe me, they'll find out eventually), the more time they have to work around it. I include an analysis of the possible solutions as well as their technical costs. It can take a lot of courage to have this discussion—but addressing a problem successfully can build trust like nothing else.
Be careful, though. It's easy to focus too much on meeting your commitments in such a way that actually damages trust. Suppose that you need a few more hours in an iteration to finish a particularly valuable story. A little bit of overtime is fine. Sprinting with a burst of more overtime can even be good for team morale and cohesiveness under the right conditions (see Energized Work in Chapter 5). Relying on overtime to meet overcommitment saps the team's energy and reduces your ability to absorb problems. Ironically, it leads to more missed commitments; you implicitly promise your stakeholders more than you can deliver for the price they expect to pay in time and resources.
Reliance on overtime indicates a systemic problem.
If you find yourself using a lot of overtime, something is wrong. Most iterations shouldn't have problems. When a problem does occur, you should usually be able to solve it by using slack, not overtime. I look for systemic problems if I see a team exceeding 10% overtime in more than one iteration per quarter.
Organizational Strategy #4: Respect Customer Goals
When XP teams first form, it usually takes individual members a while to think of themselves as part of a single team. In the beginning, programmers, testers, and customers often see themselves as separate groups.
New on-site customers are often particularly skittish. Being part of a development team feels awkward. They'd rather work in their normal offices with their normal colleagues. Those colleagues are often influential members of the company. If the customers are unhappy, those feelings transmit directly back to stakeholders.
When starting a new XP project, programmers should make an extra effort to welcome the customers. One particularly effective way to do so is to treat customers' goals with respect. This may even mean suppressing, for a time, cynical programmer jokes about schedules and suits.
Being respectful goes both ways, and customers should also suppress their natural tendencies to complain about schedules and argue with estimates. I'm emphasizing the programmers' role here because it plays such a big role in stakeholder perceptions.
Another way for programmers to take customers' goals seriously is to come up with creative alternatives for meeting those goals. If customers want something that may take a long time or involve tremendous technical risks, suggest alternate approaches to reach the same underlying goal for less cost. If there's a more impressive way of meeting a goal that customers haven't considered, by all means mention it—especially if it's not too hard.
As programmers and customers have these conversations, barriers will be broken and trust will develop. As stakeholders see that, their trust in the team will blossom as well.
You can also build trust directly with stakeholders. Consider this: the next time a stakeholder stops you in the hallway with a request, what would happen if you immediately and cheerfully took him to a stack of index cards, helped him write the story, and then brought them both to the attention of the product manager for scheduling?
It might be a ten-minute interruption for you, but imagine how the stakeholder would feel. You responded to his concern, helped him express it, and took immediate steps to get it into the plan.
That's worth infinitely more to him than firing an e-mail into a black hole of your request tracking system.
Organizational Strategy #5: Promote the Team
You can also promote the team more directly. One team posted pictures and charts on the outer wall of the workspace that showed what they were working on and how it was progressing. Another invited anyone and everyone in the company to attend its iteration demos.
Being open about what you're doing will also help people appreciate your team. Other people in the company are likely to be curious, and a little wary, about your strange new approach to software development. That curiosity can easily turn to resentment if the team seems insular or stuck-up.
You can be open in many ways. Consider holding brown-bag lunches describing the process, public code-fests in which you demonstrate your code and XP technical practices, or an "XP open house day" in which you invite people to come see what you're doing and even participate for a little while. If you like flair, you can even wear buttons or hats around the office that say "Ask me about XP."
Organizational Strategy #6: Be Honest
In your enthusiasm to demonstrate progress, be careful not to step over the line. Borderline behavior includes glossing over known defects in an iteration demo, taking credit for stories that are not 100 percent complete, and extending the iteration for a few days in order to finish everything in the plan.
These are minor frauds, yes. You may even think that "fraud" is too strong a word—but all of these behaviors give stakeholders the impression that you've done more than you actually have.
There's a practical reason not to do this. Stakeholders will expect you to complete the remaining features just as quickly, when in fact you haven't even finished the first set. You'll build up a backlog of work that looks done but isn't. At some point, you'll have to finish that backlog, and the resulting schedule slip will produce confusion, disappointment, and even anger.
Only sign up for as many stories as the team can reliably complete.
Even scrupulously honest teams can run into this problem. In a desire to look good, teams sometimes sign up for more stories than they can implement well. They get the work done, but they take shortcuts to finish it all and don't do enough design and refactoring. The design suffers, defects creep in, and the team finds itself suddenly going much slower while struggling to improve code quality.
- "Done Done"
Similarly, don't yield to the temptation to count stories that aren't "done done". If a story isn't completely finished, it doesn't count towards your velocity. Don't even take partial credit for the story. There's an old programming joke: the first 90% takes 90% of the time... and the last 10% takes 90% of the time. Until the story is totally done, it's impossible to say for certain how much is left.
The Challenge of Truth-Telling
The most challenging project I've ever coached had a tight deadline. (Don't they all?) Our end-customer was a critical customer: a large institution that represented the majority of our income. If we didn't satisfy them, we risked losing a huge chunk of vital business.
Knowing what was at stake, I made a reliable release plan our top priority. Six weeks later, we had not only implemented the first six weeks of stories, we also had a reliable estimate of our velocity and a complete estimated list of remaining stories.
It showed us coming in late—very late. We needed to deliver in seven months. According to the release plan, we would deliver in thirteen.
The project manager and I took the release plan to our director. Things went downhill. He forbade us from sharing the news with the end-customer. Instead, he ordered us to make the original deadline work in any way we could.
We knew that we couldn't make the deadline work. We didn't have enough time to add staff; it would take too long for them to become familiar with the codebase. We couldn't cut scope because we wouldn't admit the problem to our customer.
Our jobs were on the line and we tried to make it work. We ignored Brooks' Law,4 hired a bunch of programmers, and did everything we could to ramp them up quickly without distracting the productive members of the team. Despite our best efforts, we shipped defect-ridden software six months late—within a few weeks of our original prediction. We lost the customer.
We might have lost the customer even if we had told them the truth. It's impossible to say. My experience, however, is that customers, stakeholders, and executives appreciate being made part of the solution. When you demonstrate progress on a weekly basis, you establish credibility and trust. With that credibility and trust in place, stakeholders have a greater interest in working with you to balance tradeoffs and achieve their goals.
I won't involve myself in hiding information again. Schedules can't keep secrets; there are no miraculous turnarounds; the true ship date comes out eventually.
Instead, I go out of my way to present the most accurate picture I can. If a defect must be fixed in this release, I schedule the fix before new features. If our velocity is lower than I want, I nonetheless report delivery dates based on our actual velocity. That's reality, and only by being honest about reality can I effectively manage the consequences.
4"[A]dding manpower to a late software project makes it later" [Brooks], (p. 25)
Our team seems to be stuck in the "Storming" stage of team development. How can we advance?
Give it time. Teams don't progress through the stages of development in an orderly manner. One day, everything may seem rosy; the next, calamitous. Storming is necessary for the team to progress: members need to learn how to disagree with each other. You can help by letting people know that disagreement is normal and acceptable. Find ways to make it safe to disagree.
If the team seems like it hasn't made any progress in a month or two, ask for help. Talk to your mentor (see "Find a Mentor" in Chapter 2) or an organizational development expert. (Your HR department might have somebody on staff. You can also hire an OD consultant.) If there's somebody on the team who is particularly disruptive, consider whether the team would be better off if he moved to a different project.
Isn't it more important that we be good rather than look good?
Both are important. Do great work and make sure your organization knows it.
I thought overtime didn't solve schedule problems. Why did you recommend overtime?
Overtime won't solve your schedule problems, especially systemic ones, but it might help if you have a small hiccup to resolve. The key point to remember is that, while overtime can help solve small problems, it mustn't be the first or only tool in your kit.
Why bring big problems to stakeholders' attention before smaller, already-solved problems? That seems backward.
Problems tend to grow over time. The sooner you disclose a problem, the more time you have to solve it. It reduces panic, too: early in the project, people are less stressed about deadlines and have more mental energy for problems.
You said programmers should keep jokes about the schedule to themselves. Isn't this just the same as telling programmers to shut up and meet the schedule, no matter how ridiculous?
Certainly not. Everybody on the team should speak up and tell the truth when they see a problem. However, there's a big difference between discussing a real problem and simply being cynical.
Many programmers have cynical personalities. That's okay, but be aware that customers' careers are often on the line. They may not be able to tell the difference between a real joke and a complaint disguised as a joke. An inappropriate joke can set their adrenaline pumping just as easily as a real problem can.
Venting is counterproductive when there's a better way to address the root of the problem.
What if we've committed to finishing a story and then discover that we can't possibly finish it this iteration?
Mistakes happen; it's inevitable. Perhaps programmers underestimated the technical challenge of a story. Perhaps customers neglected to mention an important aspect of the story. Either way, finding the problem early and reworking the plan is your best solution. If you can't, admit your mistake and make a better plan for the next iteration. (See Iteration Planning in Chapter 8 for more about changing your plans when something goes wrong.)
Like overtime, reworking the plan shouldn't happen too often. I look for underlying systemic problems if it happens more than once per quarter.
When you have a team that works well together, you cooperate to meet your goals and solve your problems. You collectively decide priorities and collaboratively allocate tasks. The atmosphere in the team room is busy but relaxed, and you genuinely enjoy working with your teammates.
When you establish trust within your organization and with your stakeholders, you demonstrate the value of the project and your team. You acknowledge mistakes, challenges, and problems, and you find solutions instead of trying to hide them until they blow up. You seek solutions instead of blame.
Compensation practices can make teamwork difficult. An XP team produces results through group effort. If your organization relies on individual task assignment for personnel evaluation, teamwork may suffer. Similarly, ranking on a curve—in which at least one team member must be marked unsatisfactory, regardless of performance—has a destructive effect on team cohesion. These practices can transform your team into a group of competing individuals, which will hurt your ability to practice XP.
Even without harmful compensation practices, team members may not trust each other. This is a problem, but it isn't necessarily debilitating. A team that must deliver software weekly in pursuit of a common goal will learn to trust each other... or implode from the pressure. Unfortunately, I can't tell you which outcome will happen to your team.
- Sit Together
If the team doesn't sit together, it's much harder for good teamwork to occur, and if team members also don't trust each other, it's unlikely that trust will ever develop. Be careful of using XP if the team doesn't sit together.
Trust is vital for agile projects—perhaps for any project. I'm not sure it's possible to work on an agile project without it.
The Wisdom of Teams [Katzenbach & Smith], which organizational development consultant Diana Larsen describes as "the best book about teams extant".
"Developmental Sequences in Small Groups" [Tuckman] introduces the team development sequence of "Forming, Storming, Norming, and Performing".
The Trusted Advisor [Maister et. al.] is a good resource for generating organizational trust.
The Power of a Positive No [Ury] describes how to say no respectfully when it's necessary while preserving important relationships. Diana Larsen describes this ability as "probably more important than any amount of negotiating skill in building trust."
- Next: Sit Together
- Previous: Chapter 6: Collaborating
- Up: Chapter 6: Collaborating