AoAD2 Practice: Management

Book cover for “The Art of Agile Development, Second Edition.”

Second Edition cover

This is a pre-release excerpt of The Art of Agile Development, Second Edition, to be published by O’Reilly in 2021. Visit the Second Edition home page for information about the open development process, additional excerpts, and more.

Your feedback is appreciated! To share your thoughts, join the AoAD2 open review mailing list.

This excerpt is copyright 2007, 2020, 2021 by James Shore and Shane Warden. Although you are welcome to share this link, do not distribute or republish the content without James Shore’s express written permission.

Management

Audience
Managers

We help our teams excel.

Stakeholder demos and roadmaps allow managers to see what their teams are producing. But managers need more. They need to know whether their teams are working effectively and how they can help them succeed.

Unlike the other practices in this book, which are aimed at team members, this practice is for managers. It’s primarily for team-level managers, but the ideas can be applied by middle and senior managers as well. In an environment where teams decide for themselves how work will be done (see “Key Idea: Self-Organizing Teams” on p.XX), what do managers do, and how do they help their teams excel?

Most organizations use measurement-based management: gathering metrics, asking for reports, and designing rewards to incentivize the right behavior. It’s a time-honored approach to management that stretches back to the invention of the assembly line.

Measurement-based management doesn’t work.

There’s just one problem. It doesn’t work.

Theory X and Theory Y

In the 1950’s, Douglas McGregor identified two opposing styles of management: Theory X and Theory Y. The two styles are each based on an underlying theory of worker motivation.

Theory X managers believe that workers dislike work and try to avoid it. As a result, workers have to be coerced and controlled. Extrinsic motivators such as pay, benefits, and other rewards are the primary mechanism for forcing employees to do what is needed. Furthermore, Theory X managers believe, workers want to be treated this way, because they’re inherently unambitious and avoid responsibility. Under Theory X management, the design and implementation of extrinsic motivation schemes, using tools such as measurement and rewards, is central to good management.

Theory Y managers believe that workers enjoy work and are capable of self-direction. They seek responsibility and enjoy problem-solving. Intrinsic motivators such as the satisfaction of doing a good job, contributing to a group effort, and solving hard problems are the primary drivers of employee behavior. Under Theory Y management, providing context and inspiration, so workers can work without close supervision, is central to good management.

Measurement-based management is a Theory X approach. It’s based on using extrinsic motivators to incentivize correct behavior. Agile, in contrast, is a Theory Y approach. Agile team members are expected to be intrinsically motivated to solve problems and achieve organizational goals. They need to be able to decide for themselves what to work on, who will do it, and how the work will be done.

Agile requires Theory Y management.

These assumptions are built into the foundations of Agile. Theory Y management is expected and required for Agile to succeed. Theory X management won’t work. Even if you strip out the disrespect for workers, Theory X’s underlying reliance on measurements and rewards distorts behavior and creates dysfunction. I’ll explain in a moment.

The Role of Agile Management

Some managers worry that there’s no place for them in an Agile environment. Nothing could be further from the truth. Managers’ role changes, but it isn’t diminished. In fact, by delegating details to their teams, managers are freed up to focus on activities that have more impact.

Agile managers manage the work system rather than individual work. They set their teams up for success. Their job is to guide their teams’ context so that each team makes correct choices without explicit management involvement. In practice, this means team managers:2

2Thanks to Diana Larsen for her contributions to this list.

  • Make sure the right people are on the team, so that the team has all the skills needed for its work. This includes coordinating hiring and promotions.

  • Make sure the team includes the coaches it needs.

  • Mediate interpersonal conflicts, help team members navigate the chaos of change, and help team members jell as a team.

  • Help individual team members develop their careers. Mentor individuals to become future leaders and encourage team members to cross-train so that the team is resilient to the loss of any one person.

  • Monitor the team’s progress towards fluency (see the skill checklists in the introductions to Parts II-IV) and coordinate with the team’s coaches to procure training and other resources the team needs to reach fluency.

  • Procure the tools, equipment, and other resources the team needs to be productive.

  • Ensure that the team understands how their work fits into the big picture of the organization, that they have a charter (see “Planning Your Chartering Session” on p.XX), and that the charter is updated regularly.

  • Provide insights about how well the team is fulfilling their charter and how their work is perceived by stakeholders, particularly management and business stakeholders.

  • Maintain awareness of the relationships between the team and its stakeholders, and help the team understand when and why those relationships aren’t working well.

  • Advocate for the team within the rest of the organization, and coordinate with peer managers to advocate for each others’ teams. Help the team navigate organizational bureaucracy and remove impediments to their success.

  • Ensure organizational expectations around topics such as budgeting, governance, and reporting are fulfilled. Judiciously push for relaxing those requirements when it would help the team.

Measurement Dysfunction

Measurement-based management distorts behavior and causes dysfunction.

One thing you won’t see on that list: reporting and metrics. That’s because measurement-based management distorts behavior and causes dysfunction. Some examples:

Stories and story points

A team’s manager wanted to know if the team was productive, so they tracked the number of stories their team finished every iteration. The team cut back on testing, refactoring, and design so they could get more stories done. The result was reduced internal quality, more defects, and lower productivity. (Tracking capacity yields the same results. See “Capacity Is Not Productivity” on p.XX for more about this common mistake.)

Code coverage

An executive mandated that all new code be tested. Eighty-five percent code coverage was the goal. “All new code needs tests,” he said.

Good tests are small, fast, and targeted, which takes care and thought. This executive’s teams worked on meeting the metric in the quickest and easiest way instead. They wrote tests that covered a lot of code, but they were slow and brittle, failed randomly, and often didn’t check anything important. Their code quality continued to degrade, their productivity declined, and their maintenance costs went up.

Lines of code

In an effort to encourage productivity, a company rewarded people for number of lines added, changed, or deleted per day. (Number of commits per day is a similar metric.) Team members spent less time thinking about design and more time cutting and pasting code. Their code quality declined, maintenance costs increased, and they struggled with “mushroom” defects that kept popping back up after people thought they had been fixed.

Say/do ratio

Although meeting commitments is important for building trust, it isn’t a good metric. Nevertheless, one company made meeting commitments a key value. “Accountability is very important here,” they said. “If you say you’re going to do something by a certain date, you have to do it. No excuses.”

Their teams became very conservative in their commitments. Their work expanded to fill the time available, reducing throughput. Managers started pushing back on excessively-long deadlines. Now the teams had to rush their work and take shortcuts, resulting in reduced internal quality, more defects, higher maintenance costs, and customer dissatisfaction.

Defect counts

Which is easier: reducing the number of defects a team creates, or changing the definition of “defect?” An organization that tracked defect counts wasted time on contentious arguments about what counted as a defect. When the definition was too strict, the team spent time fixing defects that didn’t matter. When it was too loose, they shipped bugs to customers, hurting customer satisfaction.

Why Measurement Dysfunction is Inevitable

Rather than doing work that achieves the best result, people do work that achieves the best score.

When people believe that their performance will be judged based on a measurement, they change their behavior to get a better score on that measurement. But people’s time is limited. By doing more for the measurement, they must do less for something else. Rather than doing work that achieves the best result, they do work that achieves the best score.

Everybody knows that metrics can cause problems. But that’s just because managers chose bad metrics, isn’t it? A savvy manager can prevent problems by carefully balancing their metrics... right?

Unfortunately, no. Robert Austin’s seminal book, Measuring and Managing Performance in Organizations [Austin 1996], explains:

The fundamental message of this book is that organizational measurement is hard. The organizational landscape is littered with the twisted wrecks of measurement systems designed by people who thought measurement was simple. If you catch yourself thinking things like, “Establishing a successful measurement program is easy if you just choose your measures carefully,” watch out! History has shown otherwise. (pp. 180-181)

The situation would be different if you could measure everything that mattered in software development. But you can’t. There are too many things that are important that—although they can be measured in some ways—can’t be measured well. Internal quality. Maintenance costs. Development productivity. Customer satisfaction. Word-of-mouth. Here’s Robert Austin again:

As a professional activity that has much mental content and is not very rotable, software development seems particularly poorly suited to measurement-based management... There is evidence that software development is plagued by measurement dysfunction. (pp. 111-112)

In practice, measurements will not be comprehensive, and inhabitants of the black box will gain control of the measurement instrument to make it report what will make them look good. (p. 131)

People—particularly in software development—hate this message. We love the fantasy of a perfectly rational and measurable world. Surely it’s just a matter of selecting the right measurements!

There is no way to measure everything that matters in software development.

It’s a pretty story, but it’s a trap. There is no way to measure everything that matters in software development. The result is an endless cycle of metrics programs, leading to dysfunctions, leading to new metrics, leading to new dysfunctions.

A [manager] who commits dysfunctional acts mistakenly believes she is in a fully [measurable] situation when she is, in fact, in a partially [measurable] situation... In real settings, managers are charged with controlling activity in their areas of organizational responsibility. Unfortunately, the need for control is often interpreted narrowly as a need for measurement-based control. The [manager’s] job is then usually perceived to be the redesign of [worker] tasks to make them more measureable. (pp. 126-127)

Even when dysfunction is discovered and it is revealed that full [measurement] has not been achieved, a [manager] may still resist the conclusion that full [measurement] cannot be achieved. She may conclude instead that she simply got it wrong when she attempted the last job redesign. An unending succession of attempts at job redesign may follow, as the [manager] tries earnestly to get it right... The result is that designers of software production systems are forever redesigning, replacing old modes of control, and substituting new but structurally similar modes, with predictable lack of success. (pp. 132-133)

Delegatory Management

Even if an effective measurement system was possible, measurements are missing the point. Agile requires Theory Y management, not Theory X management, and Theory Y management is based on intrinsic motivators, not measurements and reward systems.

Rather than thinking about measurements and rewards, focus on what intrinisically motivates your team members. What do they love about their work? Is it creating something “insanely great” that customers love? Is it pushing the bounds of technical achievement? Is it being part of a high-functioning, jelled team? Or getting lost in the flow of productive work?

Whatever the motivation, inspire your teams by showing how their work will fulfill their needs. Provide them with the resources and information they need. And step back so they can take ownership and excel.

In contrast [to measurement-based management], delegation cannot produce distortion. If the customer’s value function changes, the change is immediately reflected in the effort allocation of the [worker], as long as he is aware of the change... Under delegation, workers are likely to take more initiative; they act in accordance with their own expectations instead of reacting to whatever carrot hangs before them. (p. 109)

Make measurements inconsequential

It’s not that measurements and data aren’t useful. They are! The problems arise when people think the measurements will be used to assess performance. Unfortunately, people—especially software developers—tend to be cynical about these things. It isn’t what managers say that matters; it’s what people think that causes dysfunction.

To avoid dysfunction, you have to make it structurally impossible to misuse the data.

The easiest way to do so is to keep information private to the team. The team collects the data, the team analyzes the data, and the team discards the data. They report their conclusions and decisions, not the underlying data. If nobody else sees it, there’s no risk of distortion.

If that’s not possible, aggregate the data so that it can’t be attributed to any one person. Instead of using data to evaluate subordinates, use data to evaluate yourself. This can apply to all levels of the organization. Team managers see team measures, not individual measures. Directors see departmental measures, not team measures. And so forth.

Go to gemba

If managers don’t get data about their subordinates, how do they know how people are performing? They go to gemba.

The phrase “Go to Gemba” comes from Lean Manufacturing. It means “go see for yourself.”3 The idea is that managers learn more about what’s needed by seeing the actual work than by looking at numbers.

3“Gemba” is a Japanese word meaning “the actual place [where something happened],” so “go to gemba” literally means “go to the actual place.”

To learn about your teams, go see for yourself.

Managers, to learn about your teams, go see for yourself. Look at the code. Review the UI mockups. Sit in on stakeholder interviews. Attend a planning meeting.

Then think about how you want your team improve. Ask yourself, “Why aren’t they already doing that themselves?” Assume positive intent: In most cases, it’s not a motivational issue; it’s a question of ability, organizational roadblocks, or—and don’t discount this one—the idea was already considered and set aside for good reasons that you’re not aware of. Crucial Accountability: Tools for Resolving Violated Expectations, Broken Commitments, and Bad Behavior [Patterson et al. 2013] is an excellent resource that discusses what to do next.

Ask the team

Fluent Agile teams have more information about the day-to-day details of their work than anybody else. Rather than asking for measurements, managers can ask their teams a simple question: “What can I do to help your team be more effective?” Listen. Then act.

Define goals and guardrails

Although the team owns their work, the goals of that work are defined by management. It’s okay to put requirements and boundaries in place. For example, one director needed to know that his teams were processing a firehose of incoming data effectively. He gathered together his team of managers, told them his need, and asked them to create a measurement that teams could track themselves, without fear of being judged. The director didn’t need to see the measurement; he needed to know that his teams were able to stay on top of it, and if not, what they needed to do so.

When Metrics Are Required

All too often, managers’ hands are tied by a larger organizational system. To return to Robert Austin:

The key fact to realize is that in a hierarchical organization every manager is [also measured]. Manager performance is very difficult to measure because of the intangible nature of managerial duties... her own performance is judged mostly by how well her organization—that is, her [workers]—does according to the very measurement system the [manager] installs. The [manager] has an interest, then, in installing easily exploitable measurement systems. The [manager] and [worker] quietly collude to their mutual benefit. (pp. 137-138)

Report narratives and qualitative information rather than quantitative data.

If you must report something, provide narratives and qualitative information, not quantitative measurements that can be abused. Tell stories about what your teams have done and what they’ve learned.

That may not be enough. You might be required to report hard numbers. Push back on this, if you can, but all too often, it will be out of your control.

If you have control over the measurements used, measure as close to real-world outcomes as possible. One such possibility is value velocity.

Value velocity is an actual measurement of productivity. It measures the output of the team over time. To calculate it, measure two numbers for each valuable increment the team releases: the impact, such as revenue; and the lead time, which is the number of weeks (or days) between when development started and when the increment was released. Then divide: impact ÷ time = value velocity.

In many cases, the impact isn’t easily measurable. In that case, you can estimate the impact of each increment instead. This should be done by the sponsor or key stakeholders outside the team. Make sure that all estimates are done by the same person or tight-knit team, so they’re consistent with each other.

Remember, though, that value velocity distorts behavior just like any other metric. Whichever metrics you collect, do everything you can to shield your team from dysfunction. Most metrics harm internal quality, maintenance costs, productivity, customer satisfaction, and long-term value, because these are hard to measure and tempting to shortchange. Emphasize the importance of these attributes to your teams, and—if you can do so honestly—promise them that you won’t use metrics in your performance evaluations.

Questions

What about “if you can’t measure it, you can’t manage it?”

“If you can’t measure it, you can’t manage it” is often attributed to W. Edwards Deming, a statistician, engineer, and management consultant whose work influenced Lean Manufacturing, Lean Software Development, and Agile.

Deming was massively influential, so it’s no wonder his quote is so well known. There’s just one problem: He didn’t say it. He said the opposite.

It is wrong to suppose that if you can’t measure it, you can’t manage it—a costly myth.4

4This quote is explained and put into context at The W. Edwards Demings Institute: https://deming.org/myth-if-you-cant-measure-it-you-cant-manage-it/

W. Edwards Deming

Prerequisites

Delegatory management requires an organizational culture that understands measurement dysfunction. Despite being decades old—Deming articulated the need to remove measurement-based management in at least 19825—it’s still not widely understood and accepted.

5Point 12 of Deming’s 14 Points for Management: a) Remove barriers that rob the hourly worker to pride of workmanship. The responsibility of supervisors must be changed from sheer numbers to quality. b) Remove barriers that rob people in management and engineering of their right to pride of workmanship. This means, inter alia, abolishment of the annual or merit rating and of management by objective.

Agile can still work in a measurement-based environment, but the purpose of this book isn’t to tell you what merely works; it’s to tell you what excels. Delegatory management excels, if you’re able to use it.

Indicators

When you use delegatory management well:

  • Teams feel they’ve been set up for success.

  • Teams own their work and make good decisions without management’s active participation.

  • Team members feel confident to do what leads to the best outcomes, not the best scores.

  • Team members and managers aren’t tempted to deflect blame and engage in finger-pointing.

  • Managers have a sophisticated, nuanced understanding of what their teams are doing and how they can help.

Alternatives and Experiments

The message in this practice—that measurement-based management leads to dysfunction—is a hard pill for a lot of organizations to swallow. You may be tempted by alternatives that promise to solve measurement dysfunction through elaborate balancing schemes.

Before you do that, remember that Agile is a Theory Y approach to development. The correct way to manage an Agile team is through delegatory management, not measurement-based management.

If you do look at alternative metrics ideas, be careful. Measurement dysfunction isn’t immediately obvious. It can take a few years to become apparent, so an idea can sound great on paper and even appear to work at first. You won’t discover the rot until later, and even then, it’s all too easy to blame the problem on something else.

In other words, be skeptical of any approach to metrics that isn’t at least as rigorous as [Austin 1996]. It’s based on Austin’s award-winning economics Ph.D. thesis.

That said, there are also good, thoughtful takes on Agile management. As you look for opportunities to experiment, look for opportunities that emphasize a collaborative and delegatory Theory Y approach. The resources in the Further Reading section are a good starting point.

Further Reading

Measuring and Managing Performance in Organizations [Austin 1996] was the inspiration for this practice. It presents a rigorous economic model while remaining engaging and approachable.

Turn the Ship Around! A True Story of Turning Followers into Leaders [Marquet 2013] is a gripping read, and an excellent way to learn more about delegatory management. The author describes how he, as captain of a U.S. nuclear submarine, learned to apply delegatory management with his crew.

Crucial Accountability: Tools for Resolving Violated Expectations, Broken Commitments, and Bad Behavior [Patterson et al. 2013] is an good resource for managers who need to intervene to help their employees.

Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes [Kohn 1999] is a thorough exploration of the differences between intrinsic and extrinsic motivation.

XXX Johanna Rothman, Pollyanna Pixton

XXX The Tyranny of Metrics (Jerry Z. Muller)

XXX Deming, Out of the Crisis

Share your feedback about this excerpt on the AoAD2 mailing list! Sign up here.

For more excerpts from the book, or to get a copy of the Early Release, see the Second Edition home page.

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.