Memory Latency: to Tolerate or to Reduce

It has become a truism that the gap between processor speed and memory access latency is continuing to increase at a rapid rate. This paper presents some of the architecture strategies which are used to bridge this gap. They are mostly of two kinds: memory latency reducing approaches such as employed in caches and HiDISC (Hierarchical Decoupled Architecture) or memory latency tolerating schemes such as SMT (Simultaneous Multithreading) or ISSC (I-structure software cache). Yet a third technique reduces the latency by integrating on the same chip processor and DRAM. Finally, algorithmic techniques to improve cache utilization and reduce average memory access latency for traditional cache architectures are discussed