This is such an old chestnut, I almost decided not to post it. But despite being a well-worn piece of software engineering advice, a surprising number of people don't follow it.

The rule is simple:

Prove that you need to optimize before you do it.

In modern computing environments, there's far too many interactions to accurately predict where your performance problems could be. You've got your program, runtime environment, operating system, network, database, processor cache... when you optimize for one of them, you can take a performance hit from another. That new performance hit could actually slow you down overall. For example, by caching data in your program, you're increasing the size of your working set, which may mean more processor cache misses, which could slow your system down.

Despite this, there's innumerable articles online talking about all the things you should do in order to ensure good performance. (A classic example is the advice to use Java's StringBuffer class instead of String.) Nearly all of them require sacrificing maintainability. Ignore them... for now.

Instead, write your code for maintainability. When you have performance concerns, write them down so you can come back to them later. Every few weeks, do some simple end-to-end tests to determine if your system is fast enough. (It's important to use end-to-end tests because you won't be accurately gauging performance otherwise. Those silly "do it a thousand times in a loop" tests all run in processor cache, unlike your real program.) If your system turns out to be slow, use a profiler to discover where the bottlenecks are. That's when you can apply your optimization tricks... but only to the bottleneck.

Be sure to set concrete performance goals and only optimize when you aren't hitting them. Optimization is one of those things you can waste a lot of time on, and optimizations do disasterous things to the maintainability of your code. Work with your customers or product managers to set performance goals in advance. As soon as you hit them, stop optimizing.

Following this path will make your program easier to optimize (because it's better written), give you more time to optimize (because you spend less time on micro-optimization and maintenance), and it will focus your optimization efforts. You'll have a faster program as a result.