The Concept of Memory Hierarchy in Linux

Memory hierarchy may sound like too high an abstract thing to easily understand, but the concept is actually quite simple. In this article, the concept of memory hierarchy is explained, in particular with reference to Linux systems.

Simply put, memory hierarchy is the arrangement of storage within a computer. Think of it like a pyramid. The fastest memory is at the top—and also the most scarce. Descending down, the memory gets slower, but more plentiful. The advantage of this system is that while a small number of oft-used functions can be done quickly, a large number of functions may also be accessed at a somewhat slower rate, which is very cost effective.

The memory at the top of this pyramid is referred to as registers, or processor registers. These are built within the CPU to speed up the more immediate processes, requiring only a single CPU cycle to process. They only run a few hundred bytes in size, however, they can't do too much at once. More processor registers are very expensive, but if the system will be running multiple functions at once and can't afford to be slow, then investing in this might be a good idea.

Below these processor registers in the memory hierarchy is the cache memory. There are usually multiple levels of cache memory. Mostly these serve as an in between for the top of the hierarchy and the bottom. These are denoted by L1, L2, etc. The difference between these levels is striking: whereas the top level will only require a few CPU cycles to access, the bottom may require tens or hundreds, depending on the number of levels. Things held in cache memory are typically things that are likely to be used again in the near future.

The next level of the memory hierarchy is the main memory. This typically contains multiple gigabytes, and holds things that are to be used at some foreseeable future point, but maybe not quite as often as things in cache memory.

At the very bottom of the memory hierarchy is storage. This is the slowest of the slow, but on the other hand, it can hold hundreds or hundreds of thousands of gigabytes of data. These tend to be very cheap, so large amounts of rarely accessed data may be very cost effectively stored with a very storage-heavy computer.

Continue reading on the second page to learn about the Memory Management Unit, or MMU and how Linux manages these different types of memory.

To make sure all these processes flip back and forth between all these different tiers at the same time yet don't trip over each other, there is a piece of hardware built right into most processor units, called a Memory Management Unit, or MMU. Essentially, an MMU puts up a sort of fence around areas in memory currently being used, so that other programs don't try to use it. This is analogous to a disk partition, which separates parts of storage memory.

For more information on memory, both in general and in Linux systems, check out this in-depth article on the memory hierarchy at Linux Info.

The most sparkling development that the introduction of Linux brought was a more efficient use of memory through the kernel. This kernel memory is held constantly in the higher tiers of the memory hierarchy to increase speed. Linux systems also have a very modular structure, which allows them to have only what is immediately needed in memory, as opposed to also including unnecessary functions.

Virtual memory is another important feature, though not unique to Linux systems. This technique essentially tricks a program into thinking that it has access to all the memory it could possibly need, while in fact the memory may not be physically contiguous or even nonexistent or on other levels of the memory hierarchy. This speeds up the system considerably.

There are a number of developments to make Linux systems all the more efficient with memory usage. For instance, “compressed caching” hopes to decrease the amount of communication between the levels of the hierarchy—and thus the amount of memory used - by figuring out how to keep more data in storage so less is necessary at the higher tiers, making the whole system all the more efficient. For more projects on making Linux more memory efficient, check out the Linux Memory Management project or read an article that lists and describles several important Linux memory management commands that you can use on your system.