Multi-tier data access and hierarchical memory design: Performance modeling and analysis

In modern computing environments, memory hierarchy expands from CPU registers, high speed caches, and local memory to middle-tier, network storage, web cashing, internet storage, and active networks. As the gap between processor and memory speed grows, it becomes more important to develop an analytical model to capture all these hierarchical levels and optimize system performance. A goal of a computer architect is to design a memory hierarchy that minimizes the response time of his system by maintaining a minimal cost. This requires deciding on the number, speed and size of the hierarchical layers. In this work we study the performance of systems with multi-level hierarchical memories by studying different performance metrics of the memory access such as the hit ratios, mean and variance of the access time, mean queuing time, and probability of exceeding a given access time—which helps the designer optimize his design. We use a Markov-chain model to represent the hierarchical memory, find the distribution of the memory access time, and to evaluate different performance objective functions. We study the different performance measures of the access time and we show their behavior when increasing the number and size of intermediate memory levels while maintaining a constant cost. We show that, under reasonable assumptions for the values of the parameters involved in the memory levels, the response of a long hierarchical memory system approaches power-tailed behavior as the number of memory levels increases. Our model differs from all the previous related work by being global and general and by using the P-K formula to distinguish between the memory access time and queuing time. Moreover, our model achieves higher levels of accuracy while being expandable to different architectures and applications. We focus on the hit ratio at the different memory levels and the interdependence of the hit ratios between the different levels. We finally suggest an approximation function to our different performance objective functions and give examples of systems that have been optimized.