This is one of many entries in the Change Diary: a true story of success and failure, written as it happened.

20 Feb 2002

Today was a very good day. For the last two weeks, we've been having a growing problem with fire fighting. We're in the very last stages of releasing a product, and QA keeps discovering defects. Defects are scheduled according to what the customer feels is critical. Everything critical has to be included in the next drop. Drops take two hours because our builds aren't automated. [In contrast, a current customer's build takes 40 seconds... and that includes running more than 600 automated unit tests.] We drop twice a week.

As we've gotten closer and closer to the 'final' drop, the pressure has mounted. Yesterday , we got stuff that "absolutely has to go in!" the morning of the drop. Today, my last day before the next drop, I got a few defects that "absolutely has to go in!" an hour before the end of the day.

Up until now, I haven't been reacting to this insanity well. The project manager's approach to the problem has been to ask us (actually, to tell us in the guise of asking us) to stay later, come in earlier, work on holidays, etc. Yesterday I snapped and refused. I told the project manager that we needed to solve the firefighting problem, not to ask programmers to work extra hours. No, I wouldn't be coming in early.

True, yes. Smart, no.

I realized my mistake right away and called the PM to apologize unconditionally. I'm glad I did: things were friendly today.

So why was today a very good day? Because the same situation came up today, and I handled it much better. I think my approach got us one step closer to an agile process.

As I mentioned earlier, an hour before the end of the day, the PM brought in two new defects. I dutifully wrote them on cards, got him to prioritize them, and taped them to the whiteboard. For the first time, I also estimated them, at two hours each, and told the PM that I could probably only get one done before I left.

The PM went back to his cube (on the other side of the building). A few minutes later, I got a call:

PM: Those two new issues are absolutely critical. They have to go in to the next drop.
Me: I only have time to do one before I leave. Which one's most important?
PM: Both are critical. They won't accept the build if both aren't in. They have to be in.
Me: Well, I don't see how that's possible. I've estimated them at two hours each, and I don't have that much time.
PM: Can you come in on Friday? (Friday is traditionally a holiday.)
Me: No, I'll be out of town.
PM: Okay, well, they have to be in. Can't you just do them tonight?
Me: I have to meet someone at the airport. The latest I can leave is 6:30, which doesn't give me enough time to do both.
PM: It's critical that these two things be in the drop. Can't you just come in on Friday?
Me: No, I'll be out of town.
PM: Who else is qualified to do work on these issues?
Me: Joe or John.
PM: John's wife just had a baby, and Joe's going to be at the beach on Friday. If you can't do these tonight, I'll have to call John or Joe.
Me: My estimate for these two tasks is four hours. I've estimated them in the best way I know how. I don't see how I can get them done tonight.
PM: Can you come in for just a few hours Friday morning?
Me: I'll be out of town.
PM: Where?
Me: (My home town, two hours away.)

At this point, the breakthrough occurs. The PM accepts that the schedule is fixed. He starts negotiating scope!

PM: Okay, is there any way you can get these done more quickly?
Me: Hmm. Maybe.
PM: Can you just solve the problem in this limited case, and not the general case?
Me: Yeah, I think I can do that. I'm still not sure I can get everything done in time.
PM: Okay, let me know how it goes.

It turns out that my estimates were high and I was able to get everything done with an hour to spare. That's not the thrilling part, though. The thrilling part is that:

• I didn't lose my temper as I had yesterday when faced with the PM's pressuring.
• I was able to use my estimates as a foundation for a calm, non-confrontational approach to the problem.
• The PM stopped trying to fix schedule, scope, and resources! He accepted (eventually) that the scope had to be changed, and worked with me to find a win-win solution.

Best of all, we had a brief discussion just before I left. In that meeting, the PM said he wanted to have a meeting after we delivered this version to estimate and plan all of the tasks for the next version. Perfect! It shouldn't be hard to turn that exercise into a planning game.

Years Later...

24 Mar, 2006

As I mentioned in last week's entry, pressuring us to work extra hours was the main tool our project manager used. Here, you can see him put that tool to use. The conversation quoted above really happened, almost exactly as written.

What could the PM have done differently? Unfortunately, he had little recourse. After my wife read last week's entry, she commented that she felt sorry for the project manager. She was right. After the project ended, the PM (who was a contract employee, as I was) was let go. I'm not privy to the details, but my guess is that it's because the project didn't come in on time, had a lot of defects, etc.

As I said, the PM had little recourse. By this point, the trajectory of the project had been determined. There were little things we could do to improve productivity, but the bulk of the work had been done and all that remained was to fix defects as fast as we could. All the levers available to the PM (adjusting scope, time, and resources) were stuck in place.

So, I have some compassion for what our PM was going through. Remember, it was 2002. Jobs were hard to find. The PM was scared.

That still doesn't excuse his approach. It didn't do any good--in fact, I'm sure it hurt performance--and, in my opinion, it was inhumane and unethical.

A Different Way

If I am ever in the PM's shoes, stuck at the end of a project with so many defects that it's certain to go over deadline, I will do things differently.

First, I will immediately set expectations with everybody involved: I'll tell them that we have an unknown number of defects, that we're going to be late, and that I can't say exactly when the product will be done. I'll start tracking the number of "must-fix" defects reported every day and how long it takes to fix them. I'll use that information to establish a trend and then provide a "best guess" range of completion dates to stakeholders.

Second, I will be absolutely professional and respectful in my dealings with everybody involved. With regards to the development team, I will let them know about our situation, avoid placing blame, and focus on their well-being. Stress kills programmers' ability to think clearly, and thinking clearly is essential to producing good, low-defect work. So it's not altruism when I say that I will look for ways to reduce tension and make things fun. I will ask programmers to work hard, to go home on time, and be well rested every day. I will do everything in my power to excuse them from unnecessary meetings and outside commitments. Most importantly, I will allow them to do their best work by shielding them from the inevitable complaints and pressure that will shower down from stakeholders.

None of that will help us meet the deadline. We'd probably get done a little faster, but not so much so that anybody would notice. And none of that would have saved our PM his job.

But I can't imagine doing it any other way. It's the honorable, ethical thing to do, and even if I lost my job, I would keep my self respect. I would know that I did the best job possible under difficult circumstances.

The "Soft Spurn"

Okay, before I wrap up, a few comments on the conversation I had with the PM. Coincidently, as I was preparing for this week's entry, I read Esther Derby's blog entry on the "soft spurn."

As Esther Derby summarizes it, when using the soft spurn, you:

1. Show appreciation;
2. Give a regretful "no," without making excuses; and
3. Make an opening for some future relationship.

The "without making excuses" part is important. If you make excuses, it leaves the door open for the other party to argue with you. You can see a little bit of that in the transcript of my conversation with the PM. I did a pretty good soft spurn, but I relied on the excuse of being out of town. I would have run into trouble if I had been planning on staying at home. I would say it a little better now:

PM: Those two new issues are absolutely critical. They have to go in to the next drop.
Me: I understand how important these are, so I'll stay late to work on them. That gives me time to do one of these issues. Which one would you like me to do?
PM: Both are critical. They won't accept the build if both aren't in. They have to be in.
Me: As I said, I have time to do one before the end of the day.
PM: Can you come in on Friday?
Me: I appreciate your confidence in me, and I have other plans. I'll be happy to take care of the other item for you first thing Monday.
PM: What plans?
Me: I'll be attending to some personal matters.
PM: Okay, well, they have to be in. Can't you just do them tonight?
Me: Thank you for asking, and I have other plans. I'll be available on Monday.
PM: It's critical that these two things be in the drop. Can't you just come in on Friday?
Me: As I said, I have other plans and I'll be happy to take care of it on Monday.
PM: Who else is qualified to work on these issues?
Me: Joe or John.
PM: John's wife just had a baby, and Joe's going to be at the beach on Friday. If you can't do these tonight, I'll have to call John or Joe.
Me: Okay.
PM: Can you come in for just a few hours Friday morning?
Me: I'll be back at work on Monday morning.

Same basic conversation, but without the sense that I have to defend what I do in my off hours. I also added a little emphasis, in my first sentence, that I was meeting the PM part-way.

Even with that emphasis, the soft spurn isn't necessarily a career-enhancing move, as Esther mentions. I was willing to lose my job rather than be bullied. If you're in a situation where saying "no" could jeopardize your job, giving in to the bullying and working extra hours may be your best course of action. Only you can decide. But if it were me, I'd start looking for another job immediately.

An Aside

20 Feb 2002

When I originally published this diary on Ward's wiki, people sometimes interjected questions and comments. I've kept the most interesting, along with my original answers from 2002. Here's one that was associated with this entry...

Question: If your build system is such a nightmare (two hours for a "drop", with a complicated list of instructions), why don't you fix it? On my previous assignment, we had build instructions that involved hopping from this machine to that machine to run the various commands necessary to build the system -- clumsy, fragile, and annoying. You also can run into problems where building a version of the system out of version control doesn't give you the same system, because the build rules have changed. I changed it into "push the button and it goes" system -- still a bit clumsy, perhaps, but now it's revision controlled and, most importantly, doesn't eat up a programmer hour just to compile the system.

Brent Newhall

My answer to this question is clear in my own mind, but I'm having trouble expressing it. Everybody in my organization already knows that fixing the build system is a good idea, but they're not making it a priority. I hate to drag out a hoary old cliche, but think of it this way: fixing the build system would be like giving the organization a fish. Instead, I'd like to teach the organization to want to fish.

So I'll be successful when the organization says, "We're wasting too much time on manual builds. Hey, programmers, how much time would it take to automate this?" (Programmers answer.) "Okay, do it. We should recognize return on that investment by the end of this month, so let's tentatively add these two items into the release plan for the next month." I'd even be happy if they said, "No, that's too much time. Keep doing it manually." Any answer would be fine -- I just want conscious consideration of issues like these.

To achieve this goal, I'm doing several things:

• When a drop is planned, I write it on a card and put it on the whiteboard
• I measure the time it takes to do a drop, and use that for estimates
• I frequently comment about the amount of time a drop takes
• I had this conversation in earshot of several people on the project:

  Programmer: Thanks for putting the build checklist on the wiki.
  Me: No problem.
  I've wanted it for a while, but the other guy never had time to do it.
  Me: Well, I didn't have time not to do it!
  Laughter all around, further comments (not by me) about not having time to not do things

But none of this was really planned. I'm not cooly calculating everything in the back room, manipulating people and events with precision. I'm just following my instincts and experience, reacting to situations as they arise. Until I answered your question, I didn't think about why I wasn't automating the build. I just knew that it wasn't a priority for me.

Well said. Note the use of the term priority: The build system may be horrific, but other things may be even more important. This is an extremely important principle, and is often forgotten in the rush to do The Right Thing.

Brent Newhall

Next: Week Nine: Monday