Continuous Integration on a Dollar a Day

There’s an easier, cheaper way to do continuous integration than using a build server like CruiseControl. In fact, it’s so easy, you can start doing it right this second and stop feeling bad that IT hasn’t okay’d your request for a build server yet.

(The dirty little secret? What I’m about to tell you is better than using CruiseControl!)

Step 1: Find an Old Development Computer

Find a spare box that you used to use for development. Not too old... it needs to be able to run a build. Get a crappy monitor and a spare corner. Hook it up. Put an old, broken chair in front of it. No need to get comfortable... you won’t be sitting here much.

Step 2: Get a Rubber Chicken

No, really.

Picture of a computer with a rubber chicken plugged into it

My office, circa 2001

You can use something else, like a stuffed animal, if you want. Make it fun and make sure it won’t poke anybody’s eyes out if you “accidently” toss it too hard (especially when you miss). If you don’t have anything suitable, don’t let that stop you. Improvise. Have fun.

I thought this would help on the “improvise” front, but it just made me feel incompetent, so never mind.

Step 3: Buy a desk bell

The kind that goes “ding” when you tap it. No, don’t stop because you don’t have a desk bell. Get it later. You’ve got momentum now. Crappy computer, check. Plugged-in, check. Ridiculous toy, check. You’re a few short steps away from being continuously integrationed. (If you’re really, really cheap you can skip this step entirely.)

Step 4: Automate your build

D’oh! This is the hard part. The good news: one of the best things about CruiseControl is that it makes you automate your build. The better news: continuous integration—what you’re about to do—is more than just CruiseControl’s automated builds and I’m going to help you get all of that yummy goodness. But you still need to automate your build.

Okay, this is hard, I know. If you’ve been relying on your IDE to build for you, making an automated build probably seems like a lot of work. For now, you can probably create a batch file that runs your IDE and asks it to build. That’s not going to be good enough in the long run, though, so come back and do it right.

If you have automatically-checked unit tests, like JUnit/NUnit tests, include them in the build, too.

Before you move on, go over to the crappy build computer (see step #1) and make sure the build works okay with the latest code from your revision control system. Don’t use any revision control at all? Um... okay... put down the keyboard and step away from the computer. Now, repeat after me: “Forgive me, world, for I have sinned. I will never program without revision control again. I will immediately go download TortoiseSvn, install it, and use it. I renounce all evil henceforth.” Thank you.

If a clean-running automated build takes longer than ten minutes to run, stop. You’re not ready for continuous integration yet. You’ll need to work on speeding up your build. You can do some of the stuff below, or you can use CruiseControl, but real continuous integration is not to be yours today.

Step 5: Drink the Kool-Aid®

This is absolutely, unquestionably, the 100% most important step on this list. Get all of the developers on your team together in a room.

If anybody asks, no, this isn’t a meeting. You’re going to get something done and be out in five minutes. Useful. Short. Therefore, not a meeting.

Now, without causing bodily harm, get everyone to agree to the following:

“From now on, our code in revision control will always build successfully and pass its tests.”

If somebody complains that that’s too hard, let them know that, with continuous integration, this will be easy. Um, easier. If they still think it’s too hard, politely remind them that their job is to, you know, build software.

Ooh, too harsh. Don’t say that. Yikes, I just lost ten potential customers. Oops, there goes another. Eleven.

Actually, the “everybody agreeing this is a good idea” part really is important. You see, being able to really rely on your software building, all the time, is the really revolutionary part of continuous integration. Imagine how much easier life would be if you knew that the code you just got from revision control would Just Work™.

Let me tell a little story. I maintain a piece of open-source software called NUnitAsp. Last year, I taught a course about it. During the class, somebody asked me for a feature that NUnitAsp didn’t have. I looked at the code and the change was easy. So I made the change (it took a few minutes). Then I ran my build, 96 seconds later had a new release file, and handed it out. True story. We even had an award program. We called it “find a bug, win a mug.” (We were nice: we gave everybody a mug even though they didn’t find any bugs.)

Okay, you probably won’t be there yet. You need great automated tests to build and release like that. So let me tell another story. On another project, not so successful, we all worked on different parts of the code. We were careful to check in every day or so, but we didn’t build the whole project or run tests. (Tests? We didn’t have any automated tests.) Six months later, we tried to integrate and nothing fit together. It took us a week just to get the program to run. I won’t even begin to describe how many bugs this thing had. Continuous integration, even without great tests, means you won’t ever face this nightmare again.

I hate trying to convince people this stuff is a good idea. I don’t tell you to brush your teeth, do I? Yet you do it anyway. It’s Good For You. Don’t wanna do it? Don’t do it! Not my problem.

Twelve... thirteen... fourteenfifteensixteen... crap.

Anyway, if you don’t have everybody agree to do it, then this process won’t work. What did you expect for a buck?

Step 6: Launch!

See here for a printable checklist.

You’re ready to go! Let’s run down the pre-launch checklist:

  1. Build computer? Check.
  2. Ridiculous toy? Check.
  3. Annoying bell? Optional.
  4. Automated build? Check.
  5. Group agreement? Check.

Now, let’s do it!

To start with, check in at least twice per day. That’s the “continuous” part. When you’re good at it, you’ll check in every hour or two.

Before getting the latest code from revision control, check to see if anybody has the rubber chicken. If someone does, wait for them to finish checking in.

When you check in, follow these steps:

  1. Run the build/test script locally and make sure it passes 100%.

  2. Get the rubber chicken from its resting place. If it’s not there, find the person who has it and annoy them until they’re done with their check-in.

  3. Get the latest code from the repository and run the build script again just in case. If it doesn’t pass, you know that there’s some integration problem with the code you just got. Bitch and moan, put the chicken back, and get the person who last checked-in to help you out. Start over when you’re ready.

  4. Check in your code.

  5. Walk over to the crappy build computer, get the latest code from the repository, and run the build script again. If it doesn’t pass, revert (undo) your check-in. You installed some new software, or modified an environment variable, or set a registry setting, or forgot to add a file to the repository, or something. Anyway, you need to fix the problem on your computer and try again. You can hang on to the chicken for a moment, but give it back (and start over) if anybody needs it.

  6. Ring the bell. (Everyone else, that’s your cue to applaud, or otherwise rejoice. “Yaaay.”) Put the chicken back. You’re done.

By the way, when step E fails, you’ll be tempted to just fix the problem directly on the build machine. But if you do that, the next poor slob to get the latest code won’t be able to build.

Last, but not least: Keep your build times under ten minutes. Less than five minutes is even better. If they get too long, this process will stop being a pleasant mid-morning/afternoon break and start being a real pain in the keister. Fast build times are Good For You in general, anyway, because slow builds often mean flaws in your testing approach.

Why This is Better Than CruiseControl

  1. The code in revision control always builds and passes tests. Period.

  2. If something goes wrong, you know what the problem is. It’s either your code (failed step A), integration with somebody else’s code (failed step C), or environmental changes (failed step E). You find out right away, making fixing it a lot easier.

  3. You’ll never have to fix somebody else’s broken build because they’ve left for lunch without waiting for CruiseControl to finish up.

  4. You’ll keep your build times down (nothing like painfully long integrations to motivate change), which will mean better tests, which will lead to better design.

Advanced Class

Once you’ve gotten the basics working, you can move on to really refining your use of continuous integration. One option is to make your build standalone. In other words, everything you need to build is in source control, and once you’ve gotten the code, you can disconnect from the network to build. This is good because it makes your build more reliable, allowing you to build old versions without difficulty. It also helps flush out database configuration and migration errors.

Really amazing tests are a good choice, too. If you have really amazing tests, then you can release at the drop of a hat.

I also like to make my build script build an install package. People often leave the installer out of their testing and integration practices... then suffer when it comes time to build an installation package. This is one of those things that’s much easier to do if you build it incrementally. Automatically testing installers is a pain, though.

And... I hate to admit it... installing CruiseControl can be a good idea, too. But only if you’re graduating from the advanced class. By now, you’ve gotten really good at the basics (team agreement, fast tests, never breaking the build) and you’re less likely to slip into bad practices.

Good luck! And don’t forget to send me my dollar.

Related material:

If you liked this entry, check out my best writing and presentations, and consider subscribing to updates by email or RSS.