The Importance of Cache in Modern Processors

There are two traits that a person looking to buy a new processor for their desktop immediately pays attention to - the clock speed an the number of cores. These two traits give the buyer a general idea of how much faster the processor is going to be. They're also easy to understand because their purposes are well understood. The clock speed of a processor is known to make a processor faster, and a higher number of cores is also generally assumed to make the processor, particularly in demanding applications, quicker.

But there is another important and relatively easy to understand part of the processor which often is forgotten about - the cache. The cache can be summed up in numerical format, and its size has a direct impact on the overall performance of a processor. Without considering the cache size of the processor, it is impossible to make a fully informed decision about your next processor upgrade.

The cache of a processor is a form of short-term memory which is incorporated into the processor. As you might know, the reason that a desktop computer has RAM is that accessing commonly used data from the RAM is much faster than accessing that data from a hard disk. RAM more bandwidth and has a much lower latency. The cache of the processor is extremely similar in concept. Just as the RAM exists to hold commonly used data so the computer will be responsive to common requests, the cache on a processor holds data commonly used by the processor so that frequently requested tasks can be completed more quickly.

The cache can come in multiple forms. Many modern processors use up to three levels of cache, with each level being larger than the last. This structure is implemented because of a trade-off between speed and size. Obviously, a cache with more memory will hold more data. However, the cache with more data will also take more time to access because there is more data to sort through, increasing latency. Modern processors use multiple caches in an attempt to have the best of both worlds. The first cache, called the level one cache, is the smallest. Data can be easily access from it. If the processor does find what it needs in the level one cache, then it will proceed on to the second cache, and so on. No matter how many levels of cache a processor has, they all remain integrated into the processor. It is only after the processor fails to find data in any cache that it will move on to main system memory.

The main reason cache is important is that it increases the real speed of a processor by providing the processor with data more quickly. A processor can only crank through data if it is being given data, and any delay that exists between when the processor requests data and when the processor receives it means that clock cycles are being left idle which could have otherwise been used.

That said, summing up the importance of cache in a general way is difficult to do. Cache performance has traditionally been ignored because until very recently it did not have a major impact. The cache for most Pentium 4 series processors, for example, was only 256 or 512 kilobytes, depending on the processor. Having some cache was useful, but it did not have major impact on application performance because there was not enough cache for applications to make use of. For example, the original Celeron had no cache integrated into the processor, but it still performed extremely well in games compared to processors which did include cache. This was partly because cache sizes were not large enough to hold more than extremely common and general sets of data and partly because games generally don't make much use of CPU cache because there the amount of frequently used data far exceeds what is available.

However, cache has become more important in modern processors thanks to better implementations and an increase in cache size. This increase in the important of cache is probably due to the problems that became evident in Intel's Pentium 4 line-up. The Pentium 4 processors relied heavily on high clock speeds, and as a result they were power-hungry and ran hot. Intel learned from this when it created the Core 2 Duo series, which used lower clock speeds but made up for it with multiple cores and a much larger cache. These days, it is not unusual to find a processor with an L2 cache of 8MB. Cache has become important because it gives Intel and AMD a way to increase the performance of their processors. Cache goes hand-in-hand with the trend towards processors with multiple cores. Both the Phenom II and the Core i7 possess larger cache sizes then the Phenom and the Core 2 architectures, and the trend of increasing cache size is likely to continue.

While cache is an important part of a modern processor, it differs from desktop or video card RAM in that it does not have the same bottleneck effect when too little is available. As many users know from experience, trying to use a program that demands 512 megabytes of RAM on a desktop that only has 256 megabytes installed is frustrating, if not impossible. The same applies to video RAM. It is entirely possible to use a processor that has an extremely small cache and be satisfied by its performance. Such a processor will not hold back your PC in the same way that having to little RAM will hold back your PC. This means that there is no real "minimum" amount of cache that is needed.

That said, the cache provides a general boost to processor speed that is helpful in almost all situations. Tom's Hardware once performed a test on different processors that had different cache sizes but were over-clocked and under-clocked so that they ran at the same clock speed. The conclusion of the test was that one megabyte of cache tended to offer a boost equal to one clock speed increment. So, a processor with a clock speed of 2.4 Ghz and 4 megabytes of L2 cache would theoretically perform the same as a 2.6 Ghz processor with 2 megabytes of cache. This is obviously an estimate, and it is likely impossible to absolutely nail down the performance increase that results from a larger cache.

But the increase is there, and it is important to keep in mind when buying your processor. For example, reviewers of the AMD X3 720 have taken note that in some applications it seems to repeatedly report benchmark results that are equal to or better than the X4 920 or X4 940. This extra performance is probably thanks to the larger, faster cache on the X3 720. As a result, the X3 720 provides better performance than similar products with a higher clock speed and a higher price tag, and that means you can get more bang for your buck. And who doesn't love that?