Content-aware memory systems for high-performance, energy-efficient data movement

Thesis (Ph. D.)--University of Rochester. Department of Computer Science, 2017.Power dissipation and limited memory bandwidth are significant bottlenecks in virtually all computer systems, from datacenters to mobile devices. The memory subsystem is responsible for a significant and growing fraction of the total system energy due to data movement throughout the memory hierarchy. These energy and performance problems become more severe as emerging data-intensive applications place a larger fraction of the data in memory, and require substantial data processing and transmission capabilities. As a result, it is critical to architect novel, energy- and bandwidth-efficient memory systems and data access mechanisms for future computer systems. Existing memory systems are largely oblivious to the contents of the transferred or stored data. However, the transmission and storage costs of data with different contents often differ, which creates new possibilities to reduce the attendant data movement overheads. This dissertation investigates both content aware transmission and storage mechanisms in conventional DRAM systems, such as DDRx, and emerging memory architectures, such as Hybrid Memory Cube (HMC). Content aware architectural techniques are developed to improve the performance and energy efficiency of the memory hierarchy. The dissertation first presents a new energy-efficient data encoding mechanism based on online data clustering that exploits asymmetric data movement costs. One promising way of reducing the data movement energy is to design the interconnect such that the transmission of 0s is considerably cheaper than that of 1s. Given such an interconnect with asymmetric transmission costs, data movement energy can be reduced by encoding the transmitted data such that the number of 1s in each transmitted codeword is minimized. In the proposed coding scheme, the transmitted data blocks are dynamically grouped into clusters based on the similarities between their binary representations. Each cluster has a center with a bit pattern close to those of the data blocks that belong to that cluster. Each transmitted data block is expressed as the bitwise XOR between the nearest cluster center and a sparse residual with a small number of 1s. The data movement energy is minimized by sending the sparse residual along with an identifier that specifies which cluster center to use in decoding the transmitted data. At runtime, the proposed approach continually updates the cluster centers based on the observed data to adapt to phase changes. By dynamically learning and adjusting the cluster centers, the Hamming distance between each data block and the nearest cluster center can be significantly reduced. As a result, the total number of 1s in the transmitted residual is lowered, leading to substantial savings in data movement energy. The dissertation then introduces content aware refresh—a novel DRAM refresh method that reduces the refresh rate by exploiting the unidirectional nature of DRAM retention errors: assuming that a logical 1 and 0 respectively are represented by the presence and absence of charge, 1-to-0 failures dominate the retention errors. As a result, in a DRAM system that uses a block error correcting code (ECC) to protect memory from errors, blocks with fewer 1s exhibit a lower probability of encountering an uncorrectable error. Such blocks can attain a specified reliability target with a refresh rate lower than what is required for a block with all 1s. Leveraging this key insight, and without compromising memory reliability, the proposed content aware refresh mechanism refreshes memory blocks with fewer 1s less frequently. In the proposed content-aware refresh mechanism, the refresh rate of a refresh group—a group of DRAM rows refreshed together—is decided based on the worst case ECC block in that group, which is the block with the greatest number of 1s. In order to keep the overhead of tracking multiple refresh rates manageable, multiple refresh groups are dynamically arranged into one of a predefined number of refresh bins and refreshed at the same rate. To reduce the number of refresh operations, both the refresh rates of the bins and the refresh group-to-bin assignments are adaptively changed at runtime. By tailoring the refresh rate to the actual content of a memory block rather than assuming a worst case data pattern, the proposed content aware refresh technique effectively avoids unnecessary refresh operations and significantly improves the performance and energy efficiency of DRAM systems. Finally, the dissertation examines a novel HMC power management solution that enables energy-efficient HMC systems with erasure codes. The key idea is to encode multiple blocks of data in a single coding block that is distributed among all of the HMC modules in the system, and to store the resulting check bits in a dedicated, always-on HMC. The inaccessible data that are stored in a sleeping HMC module can be reconstructed by decoding a subset of the remaining memory blocks retrieved from other active HMCs, rather than waiting for the sleeping HMC module to become active. A novel data selection policy is used to decide which data to encode at runtime, significantly increasing the probability of reconstructing otherwise inaccessible data. The coding procedure is optimized by leveraging the near memory computing capability of the HMC logic layer. This approach makes it possible to tolerate the latency penalty incurred when switching an HMC between active and sleep modes, thereby enabling a power-capped HMC system