Locality of Reference and Cache Operation in Cache Memory

This article deals with the concept of locality of reference and cache operations in cache memory. Locality of reference refers to the process of accessing a particular portion of memory at any given time period. Cache memory is a small-sized memory that allows fast access by storing the frequently needed data and instructions. And cache operations are those operations that allow to store data and instructions in cache memory so that fast retrieval is possible.

What is Cache Memory?

Cache memory is a small, high-speed storage area in a computer. The cache is a smaller and faster memory that stores copies of the data from frequently used main memory locations. There are various independent caches in a CPU, which store instructions and data. The most important use of cache memory is that it is used to reduce the average time to access data from the main memory.

By storing this information closer to the CPU, cache memory helps speed up the overall processing time. Cache memory is much faster than the main memory (RAM). When the CPU needs data, it first checks the cache. If the data is there, the CPU can access it quickly. If not, it must fetch the data from the slower main memory.

What is Locality of Reference?

Locality of reference refers to a phenomenon in which a computer program tends to access same set of memory locations for a particular time period. In other words, Locality of Reference refers to the tendency of the computer program to access instructions whose addresses are near one another. The property of locality of reference is mainly shown by loops and subroutine calls in a program.

1. In case of loops in program control processing unit repeatedly refers to the set of instructions that constitute the loop.
2. In case of subroutine calls, every time the set of instructions are fetched from memory.
3. References to data items also get localized that means same data item is referenced again and again.

In the above figure, you can see that the CPU wants to read or fetch the data or instruction. First, it will access the cache memory as it is near to it and provides very fast access. If the required data or instruction is found, it will be fetched. This situation is known as a cache hit. But if the required data or instruction is not found in the cache memory then this situation is known as a cache miss. Now the main memory will be searched for the required data or instruction that was being searched and if found will go through one of the two ways:

1. First way is that the CPU should fetch the required data or instruction and use it and that’s it but what, when the same data or instruction is required again. CPU again has to access the same main memory location for it and we already know that main memory is the slowest to access.
2. The second way is to store the data or instruction in the cache memory so that if it is needed soon again in the near future it could be fetched in a much faster way.

What is Cache Operation?

Cache operations are those operations that allow fast data retrieval by using principle of locality of reference. They store the nearby or same data in the cache memory if they are frequently accessed by the CPU. There are two ways with which data or instruction is fetched from main memory and get stored in cache memory. These two ways are the following:

1. Temporal Locality – Temporal locality means current data or instruction that is being fetched may be needed soon. So we should store that data or instruction in the cache memory so that we can avoid again searching in main memory for the same data. When CPU accesses the current main memory location for reading required data or instruction, it also gets stored in the cache memory which is based on the fact that same data or instruction may be needed in near future. This is known as temporal locality. If some data is referenced, then there is a high probability that it will be referenced again in the near future.
2. Spatial Locality – Spatial locality means instruction or data near to the current memory location that is being fetched, may be needed soon in the near future. This is slightly different from the temporal locality. Here we are talking about nearly located memory locations while in temporal locality we were talking about the actual memory location that was being fetched.

Cache Performance: The performance of the cache is measured in terms of hit ratio. When CPU refers to memory and find the data or instruction within the Cache Memory, it is known as cache hit. If the desired data or instruction is not found in the cache memory and CPU refers to the main memory to find that data or instruction, it is known as a cache miss.

Hit + Miss  = Total CPU Reference
Hit Ratio(h) = Hit / (Hit+Miss) 
Miss Ratio = 1 - Hit Ratio(h)
Miss Ratio = Miss / (Hit+Miss)

Average access time of any memory system consists of two levels: Cache and Main Memory. If Tc is time to access cache memory and Tm is the time to access main memory then we can write:

Tavg = Average time to access memory

For simultaneous access
Tavg = h * Tc + (1-h)*Tm  

For hierarchial access
Tavg = h * Tc + (1-h)*(Tm + Tc) 

Conclusion

In conclusion, locality of reference is an important concept that tells us how to utilize the cache memory efficiently in such a way that increases the speed of data retrieval. This is possible due to the cache operations – Temporal locality and Spatial locality which help to store the frequently accessed data in the cache memory. Thus, cache performance increases which can be measured by finding the average memory access time.