Update, 15 Jan 2008: Note that the term "customer" in this essay is used in the agile sense of the term--that is, they're on-site customers. Business experts, product owners, and so forth. They're not the people actually funding development.
"We'd like to constantly improve our codebase," the programmers said. I'd just explained the importance of filling in gopher holes and paying down design debt. "But our customers won't let us! They insist that we take the quickest way every time. We're an agile team, and the customers are in charge. We don't want to build up technical debt, but we don't have a choice."
These programmers made a crucial mistake. They forgot their professional responsibility.
Misunderstanding Agile Planning
Agile development puts customers first. Rather than build software by technical priorities, teams build software in order of business priorities. Business experts--"on-site customers"--are said to "steer" the development effort.
Actually, this isn't entirely true. It's a convenient shorthand for what's really going on. Customers steer, yes, but they're not entirely in charge.
The goal is to balance high value with low cost.
What's really happening in agile planning--let's take XP's planning game, to be specific--is that both customers and programmers are combining their expertise to create the best possible plan. Customers contribute expertise about business value and programmers contribute expertise about development costs. The goal is to create a plan that balances high value with low cost.
Practically speaking, this means that programmers provide estimates for each story and customers decide the order in which stories will be developed. That's why we say "customers steer."
But there's a critical point here, one that's easily missed. Customers get to say when stories are developed. They even get to say what stories are developed. But they don't get to say how!
Restore the Balance
In the bad old days of waterfall development, programmers would figure out requirements, create a design, and then implement the design in the way that was most technically convenient. Programmers were king. Plans were created entirely by programmers. Or the programmers' Big Boss set a deadline--perhaps motivated by political pressures, or by romanticized memories of what he could program over a Jolt-fueled weekend--and programmers scrambled to meet it.*
*Yes, I'm over-simplifying a complex situation. I'm Allowed.
Agile development reacted to this, saying, "Hey! Wait a second! This isn't working! We're not delivering value to our customers." The planning game restores balance to the equation. Customers are in charge of value; programmers are in charge of cost.
Some agile teams have let the pendulum swing too far. They've abdicated responsibility. They say, "The customers are in charge! We have to do whatever they say."
That's taking it too far.
Cost Is Our Responsibility
Software rewrites are hideously expensive. Not only do you spend an arm and a leg reproducing features that you already have, you stop producing any new features while you do so. It's a huge opportunity cost. Your software slips steadily behind the times and your competitors have a field day.
Yet software barely lasts five years before programmers start pushing for a rewrite. I made this point to a friend recently, and he looked at me blankly. "Isn't that just the way software is?"
Software doesn't have to degrade over time.
No! It doesn't have to be that way. Software doesn't have to degrade over time. It can get better. It's not even very hard: just make the software better every week. Even if only a little bit.
This is where the problem arises. Customers always want software faster than we can deliver it. It's like a law of nature. The problem is... we give in! We take shortcuts and deliver features more quickly. But we do so at the cost of adding technical debt to the software. It will take the next person who looks at the code a little more time to figure it out. It's a little bit more likely to harbor a time-sucking Heisenbug. It's a little harder to build and test.
Do this week after week, and before long, you'll spend more time struggling with the rot in your codebase than you'll spend on coming up with elegant designs and interesting algorithms. We've all been there. We've all cursed the person who came before us. Problem is... he's us.
Accept Your Responsibility
Customers don't want a rewrite. They don't want development to slow down every week. They just don't know that they're asking for these things. So next time a customer pushes you to finish a feature faster, be friendly but firm. "No problem! Let's figure out what to cut."
And when someone says, "Can't you take a shortcut and get this done faster?" look them straight in the eye and say "No." "No, we can't do that without hurting our schedule. But I can help you find ways to simplify features so we can finish faster."
Don't hem. Don't haw. And above all, don't argue about it. You're the expert. You're the one developing the application in the most cost effective way possible. You have a professional responsibility to do your job well.
So do it.
Danger! Danger, Will Robinson!
Please don't take this essay as permission to go off on wild flights of fancy in the name of "quality." Please don't tell your customers that you're going to refactor for the next month and they don't have a say in the matter. That would be irresponsible, not to mention a really good way to kill your relationship with your customers. You need to figure out how to deliver high-quality code in an incremental fashion while continuing to meet customer needs. XP has lots of good things to say about this subject, and Shane and I cover it (plug, plug) in our book. See chapter 9.
PS: Bob Martin recently wrote about this subject in "Business Software is Messy and Ugly."