If you are economizing, here's a simple rule:
Do buy a CPU/motherboard one or two levels lower than commercial state of the art.
In February 2004, the PC market is all Pentium IV and AMD Athlon chips, speeds ranging from 2.2 to 3.4GHz. For best value, look at the middle of that range.
Why? Because of the way manufacturers' price-performance curves are shaped. The top-of-line system is generally boob bait for corporate executives and other people with more money than sense. Chances are the system design is new and untried — if you're at the wrong point in the technology cycle, the chip may even be a pre-production sample, or an early production stepping with undiscovered bugs like the infamous FDIV problem in early Pentiums. You don't need such troubles. Better to go with a chip/motherboard combination that's been out for a while and is known good. It's not like you really need the extra speed, after all.
Besides, if you buy one of these gold-plated systems, you're only going to kick yourself three months later when the price plunges by 30%. Further down the product line there's been more real competition and the manufacturer's margins are already squeezed. There's less room for prices to fall, so you won't watch your new toy lose street value so fast. Its price will still drop, but it won't plummet sickeningly.
Again, bear in mind that the cheapest processor you can buy new today is plenty fast enough for Linux. So if dropping back a speed level or two brings you in under budget, you can do it with no regrets.
Another easy economy measure is looking for repaired or reconditioned parts with a warranty. These are often as good as new, and much cheaper. (This is an especially good tactic for monitors and hard drives.)
Your display is one of the areas where pinching pennies is not a good idea. You're going to be looking at that display for hours on end. You are going to be using the screen real estate constantly. Buy the best quality, largest screen you possibly can — it will be worth it.
There is a fair amount of price variance among equivalent video cards, so shop aggressively here. We won't do this, but if you're on a budget, one easy thing to trade away is bit depth. Manufacturers like to include 24- and 32-bit "photographic" color as sizzle in their advertisements, but unless you're doing something like specialty photocomposition work or medical graphics you'll probably never use more than 65535 colors. So you can settle for 16-bit color (used to be you could settle for 8-bit, before websites started routinely stepping outside the 216-color "web-safe" palette).
On the other hand, you probably don't need the latest and greatest CD-ROM device. High-speed CD-ROMS are really designed for people playing CD-ROM games or other applications involving image and sound archives. If you're doing the Linux thing, chances are you'll primarily use CD-ROMs that are code archives. Your average transfer size will be small and an apparent speed of 6x or even 4x quite satisfactory. So, if you need to, here's a place to cut costs by buying well behind the leading edge.