Print

Work in Progress

11 Jun, 2008

Commentary on The Art of Agile Development's Reporting practice.

As my career has progressed, I've spent more and more time with senior software managers. One thing my younger self would have been surprised to learn is that executives aren't really out to get you. Instead, they're generally interested in seeing individuals grow and succeed.

At the same time, executives have a lot to keep track of. There's no way for them to have the same insight into your work that your team does. And yet the people doing the work--the ones in the know--are often uncomfortable sharing their opinions with senior managers. Information gets transmogrified, and sometimes whitewashed, on its way to the executives' ears.

So one of the common things I'm seeing in executives is an interest in software metrics. They want to know how their teams are performing. Not so they can punish and reward people, necessarily, but so they can know where to direct resources for improvement.

This is a challenge. For one thing, it's really hard to measure productivity. Martin Fowler has said it can't be done. I've said it can be, but the metric I proposed is very difficult to measure. I'm experimenting with an alternative that's easier to measure, but the jury's still out on how well it will work.

To make things even more challenging, some people say that measurements, no matter how accurate, risk dysfunction. In "Mad About Measurement," Tom DeMarco writes:

Encouraging people to work to the numbers is what the 1960s called Management By Objectives (or MBO). MBO had a brief history as the fad du jour through the early 1970s, and then largely disappeared from the popular business press, though unfortunately not from all our organizations. The problem was that MBO didn't work very well. Companies that tried it didn't prosper; most people who used the scheme were clearly aware of the growing dysfunction it caused.

I think you can see where I'm heading: directly toward my third Disquieting Thought About Software Metrics:

DT #3: Have we in the software industry through the use of our software metrics programs inadvertently rediscovered Management By Objectives?

I suspect we have. We've stumbled upon a bankrupt concept from the past, and tried to make it work without ever understanding why it didn't work before.

Tom DeMarco, "Mad About Measurement," Why Does Software Cost So Much? (Dorset House, 1995)

Robert Austin agrees. He's the author of Measuring and Managing Performance in Organizations, based on an economic model developed in his Ph.D. thesis. The book's a bit dry, but it makes a compelling case for the problems of measurement-based management.

[O]ften the best that can be [achieved] through measurement, even after job redesign, is partial supervision [that is, only some aspects of the work can be measured], which generates relatively weak improvements in value to the customer compared to [the alternatives]. In addition, partial supervision produces significant distortion of incentives. If motivation theorists are correct in their conclusions, distorted incentives obscure inclinations [a worker] may have toward doing the job right for internal reasons, like pride in his work. Worse, if targets are set badly, distortion may become significant enough to produce dysfunction.

Robert Austin, Measuring and Managing Performance in Organizations (Dorset House, 1996)

And, as Robert Austin mentions, there's the problems that arise from using extrinsic motivators rather than intrinsic motivators.

It's a pickle. Executives have a real need for good performance measurements. But there's compelling arguments that say good performance measurements don't exist... and even if they did, they could lead to dysfunction and decreased motivation.

A tough situation. I don't have any answers yet, and I'm not ready to give up.