This is one of many entries in the Change Diary: a true story of success and failure, written as it happened.

Tuesday

16 Apr, 2002

I've been going through a lot of ups and downs lately. I alternate between extreme burn-out and calm patience. The paper is making some rounds... I'll be talking to someone with influence about it tomorrow. This is actually the first person who has questioned the things I've said, which I take to be a good sign: it means he's actually reading it, not just smiling and nodding.

Thursday

18 Apr, 2002

As I mentioned in Tuesday's entry, I talked to someone with influence over process about my paper yesterday. The upshot was that he and I have the same basic perspective on process, although I'm a bit more radical when it comes to agile development. We agree on what the major problems facing the organization are. We even agree, more or less, on the first step.

Unfortunately, that's the point where I hit a stone wall. I don't know what it is about this company: everybody happily agrees with me, but that's as far as it goes. Nothing happens. I pushed and pushed for action... gently... and the response was essentially this: "There's nothing you can do to help at this time. Please wait an indefinite time for an indeterminate result. At some point we'll let you know, and you'll have 'the important conversation' with the guy in charge."

At first, I was very disappointed with the lack of results. I still go through times every day where I feel completely helpless and burned-out. But now I think I'm just going to sit, wait, and see what happens. Communication failures are already starting to occur in a big way. We had some problems today with people accidently overwriting each others' files... exactly the kind of thing I predicted in my paper. (Unfortunately, I just predicted communication problems, not this specific issue. I wish I had!)

I hope people will connect my predictions with what's happened, but I get the impression that people think I'm writing these papers from some sort of idealistic viewpoint, not a practical one. I might say "I told you so!" eventually, but not just yet. I've bruised enough egos with my recent push. Now it's time to just put my head down and observe. I'm going to focus on low-level, tech-oriented change for a while. I've started introducing unit tests for our UI components.

I may be petty, but I'm going to enjoy watching things fall down for a while. I did everything I could to stop it, but I was ignored. I don't feel the slightest bit guilty about my enjoyment.

Maybe I do feel a bit bitter.

Friday

19 Apr, 2002

A couple of notable successes today. First, and most enjoyable: I successfully created some unit tests for the UI portion of our app. Although this may not have much impact on the organization, it felt good to write some quality code for a change.

Second, management has just done a re-org and is moving people around. This involves shuffling cube walls. I got word today that the new floor plan includes the bullpen I've been pushing for. I'm not as thrilled as I could be, though, because the project managers also moved... to a different floor. And the new bullpen won't have room for all of the programmers on the project. And QA won't be included. Sigh. It's actually not much different from what we already have.

But I can keep writing quality code.

Years Later...

25 May, 2006

For the last six-plus years, my consulting business has centered around introducing teams to agile techniques. (That's starting to change; lately, I've been getting calls from teams that already use agile practices.) As a result, I pay a lot of attention to how and when teams introduce different agile techniques. It seems to break down into three basic approaches:

1. The "Switchover" Approach. Pick an agile method that covers both structure and technology and do it all. Extreme Programming exemplifies this approach.

2. The "Structure First" Approach. Change the way projects are organized and managed and let the technology catch up later. Scrum exemplifies this approach.

3. The "Technology First" Approach. Change the techniques used to develop software and let management catch up later. There's no defined methodology that exemplifies this approach, but people tend to cherry-pick technical practices from XP.

My early experiences with agile development were in the "switchover" style and they were very successful. As time went on, I began to see flaws with the switchover approach: it can be a lot for an organization to digest. Still, it remains my favorite approach if necessary preconditions are met.

At the time I wrote the diary, I had only been involved in "switchover" projects. Out of necessity, I was taking an incremental approach, but what I really wanted was to run a pilot "switchover" project. I didn't have a lot of faith in the incremental approach and I felt that a pilot project would clearly demonstrate the value of agile development.

Given what I knew at the time, it was a sound strategy. (In hindsight, it was a bad strategy, but that's a discussion for another day.) It explains why I focused so heavily on trying to accomplish larger organizational change. I could have had plenty of success with small, technical, fixes--automate the build here, do some test-driven development there--but I didn't think that those changes would result in any significant organizational change.

Now that I have several more years of experience under my belt, can I say which of these approaches is best? No. It depends on your situation. Let's take a look at each approach.

Switchover

The "switchover" approach is still my favorite. It doesn't work for everyone, though. When I do a switchover, I use XP as my methodology and I look for these things:

• A friendly, open-minded, and cohesive team
• with supportive management
• led by at least one experienced developer (who
• groks object-oriented design and architecture)
• and an experienced product manager (who
• will sit with the programmers and
• devote his/her full attention to the project)
• ready to make a sincere effort
• to understand and apply all of the XP practices
• to a brand-new, green-field project.

That's a lot to ask. I admit it. Not all of these points are absolutely necessary, but some are. If they're all present, then a switchover to XP should be a slam dunk. If not... it depends. It depends on what isn't present and on how that lack can be accommodated.

There's so much benefit to doing all of the XP practices together that I make a extra effort to get these items in place. That's essentially what I was doing on the project I described in the diary. I made some little changes to the project I was working on, but most of my effort was directed at getting the management support in place for the next project.

(I didn't have the above list back then, of course. I was working from instinct and experience, not a tidy list. Somewhere in my brain was the prototype of the list, though. There are items on there now that weren't there in 2002. That lack was going to hurt, as we'll see.)

At any rate, you probably don't have all of these items in place. Someday I'll write an article that goes over this list in detail. For now, though, here's a brief discussion of each point.

• A friendly, open-minded, and cohesive team
• with supportive management
• ready to make a sincere effort
• to understand and apply all of the XP practices

I think these four points are absolutely crucial. Without them, I wouldn't even start with a switchover approach. "Supportive management," by the way, goes a lot further than it sounds. If you're going to introduce XP, you need an open workspace: somebody needs to wrangle the details with facilities. The switchover might make political waves. There may be impacts on other projects or external customers. Lots of little problems will occur and people need to have the patience to see them through.

• led by at least one experienced developer (who
• groks object-oriented design and architecture)

You can hire a coach to help you with these things. A smart but inexperienced team might be able to make it through on their own.

• and an experienced product manager (who
• will sit with the programmers and
• devote his/her full attention to the project)

A product manager is essential. While the technical success of the product relies on the development team, the market success of the product relies on the product manager. The better the product manager understands the product, the closer he or she sits to the developers, and the more time he or she devotes to the project, the greater the likelihood of market success. (This includes internal projects, too; their "market" is the internal users and stakeholders.)

The participation of a good product manager is one of the most difficult things to achieve, by the way. For whatever reason, qualified product managers are hard to come by.

• to a brand-new, green-field project.

A green-field project makes technical practices like test-driven development much easier. The switchover approach will also work on existing codebases, but there might be a big slowdown depending on the quality of the design.

Structure First

The "structure first" approach has gained a lot of popularity recently. Scrum is the main approach people seem to be using, and it's a good approach.

At its most basic, the structure-first approach puts the team in a room together, puts a strict timebox around the work, and asks the team to dictate how much work they can accomplish, as a group, in each timebox. In Scrum, the timeboxes are called "sprints" and they're one month long. Most people call the timeboxes "iterations." (A timebox, by the way, is a specific amount of time to accomplish some work. At the end of the time, any work not done is rescheduled.)

There is an amazing power to this approach. I've used it before with great results. One team I worked with was overwhelmed with change requests. We put in a daily timebox and saw immediate results. Productivity went up, morale went up, and the team actually found itself with spare time.

The structure-first approach is a lot less intimidating than the "switchover" approach and I think that's why it's so common. An XP switchover has the same basic structure but cranks up the intensity with one week iterations and a lot of prescriptive technical practices.

On the downside, structure-first leaves a lot out. In iterative development, you develop each technology layer incrementally and you can't be certain of the requirements for the next iteration. To compensate, design must be done differently than usual. The shorter the iterations, the more important this becomes. XP recognizes this and its prescriptive technical practices are focused on making rapid iterative development feasible. The structure-first approach doesn't address the design issue and risks accumulating more design debt as a result.

The other downside is that structure-first requires as much management support as the switchover approach. A good product manager remains critical. Team buy-in, on the other hand, is easier because there are a lot fewer practices to swallow.

The reasoning behind the structure-first approach is that it will lead to immediate improvements that will allow the team to self-organize and take ownership of its technical practices. As the pressure cooker of short timeboxes make the shortcomings of traditional technical practices clear, the team will adapt its practices accordingly.

It's a good theory and I hear of a lot of teams that are doing "Scrum with XP practices." When I look more closely, though, they don't have all of the XP practices in place, and they don't seem to have anything else in place to fill the gaps. They're making it work, but full XP is more effective.

If my goal was a full, XP-like process, and I had the management (and product management) support to do a structure-first approach, I'd take a hard look at doing a switchover instead. It's harder to get going because there are a lot more practices to swallow. Some of them, like pair programming, are difficult for teams to accept. But a switchover gets the full process in place right away, then incrementally adjusts it, rather than putting half a process in place and expecting the team to incrementally find and fill gaps on their own. Structure-first teams take a lot longer to get to a full solution and some don't make it at all.

On the other hand, if the conditions necessary for a switchover aren't in place, structure-first may work where switchover wouldn't.

Technology First

The "technology first" approach is popular with teams that don't have management support for a structural change. It's my least favorite approach. While it's easy from a change perspective, I haven't seen it accomplish much.

That's not to say that the technology-first approach is bad. In a nutshell, technology-first means adopting technical practices, typically from XP, that can be applied within the existing structure. Continuous integration is a great example. Test-driven development is another common choice.

The nice thing about technology-first is that it seems like it can be applied without convincing anybody else to going along with the idea. Just install a continuous build server, and zzzap! Continuous integration.

The problem is, that doesn't work. If you're the only person on the team that cares about continuous integration, you're going to be the one stuck fixing the build. Everyone else is going to ignore build failures. If you're the only one who cares about test-driven development, the other developers aren't going to run your tests before they check in. You'll be stuck fixing, and writing, all of the tests.

This is the worst-case scenario, of course. But when I talk to developers about organizational change, I often find that they don't even have developer buy-in, let alone management buy-in. They want change to happen quickly and they start doing one of the technical practices (usually continuous integration or test-driven development). Then they end up policing the developers, complaining whenever somebody else breaks the build. Not the best way to get the team on your side.

Technology-first, done right, involves the whole team agreeing on specific technology practices to adopt. As a group, they commit to achieving certain standards. (Continuous integration, for example: "The latest code in our source code repository will always successfully build and pass tests.")

Even so, I have not yet seen technology-first achieve significant organizational impact. If I didn't have management support for structure-first or switchover, I would focus on getting management support. At best, technology-first could provide some ammunition for those discussions, but be careful: some technical practices, such as test-driven development, take a while to show dividends. They slow you down in the meantime. Depending on your situation, they could hurt your cause rather than help.

In Conclusion

I've spent several thousand words telling you what you already knew: there is no magical solution to change. A technology-first approach might make parts of your life easier, but the big changes require supportive management. If you're going to go to the trouble of getting supportive management, and if you have a green-field project, good understanding of the technical practices, and team buy-in, you might as well do a switchover. Otherwise, structure-first could be a better bet.

There is one very important alternative that I haven't mentioned. Can you guess?

It's to do nothing at all. It takes a certain amount of hubris to decide to change an organization. It's easy to forget that, although doing nothing means "do no good," it also means, "do no harm."

Next: Week Seventeen: Wednesday