To get a sense of what kind of performance improvement we can expect going from 2 to 4 CPU cores, let's focus on the Core 2 Duo E6600 and Core 2 Quad Q6600 processors. These 2.4 GHz CPUs are identical in every respect, except for the number of cores they bring to the table. In a recent review, Scott Wasson at the always-thorough Tech Report presented a slew of benchmarks that included both of these processors. Here's a quick visual summary of how much you can expect performance to improve when upgrading from 2 to 4 CPU cores.
What's that? Software written in the days before cheap multi-core machines were readily available to programmers... aren't programmed to take advantage of multi-core? Shocker!!
I'm not saying that multi-core coding is easy. It isn't. Even the smartest programmers screw it up all the time. The human mind is set up to think procedurally. That's why even experts at lock-based programming still cause race conditions, deadlocks, and other forms of screwage with regularity. We programmers don't really understand parallel coding yet, and we don't have all the tools we need to do it well. I especially don't (yet) see any kind of reason to believe that parallelization of code can be made automated except in the simplest possible cases. Some human being is always going to have to be decide how a program splits up its tasks, and write the code accordingly.
So if you're going to buy a computer tomorrow, or next week, getting a faster 2-core may indeed be a smarter move than getting a slower 4-core. But I think in the long run, there's no contest. Programs that are properly written for N cores do see almost an N times speedup. And the tools to write code for multi-core are getting better all the time. It will probably take ten years for programmers to get fully up to speed on parallel computing. But once we do, watch out...