Data Reorganization in Memory Using 3D-stacked DRAM

In this paper we focus on common data reorganization op-erations such as shuffle, pack/unpack, swap, transpose, and layout transformations. Although these operations simply relocate the data in the memory, they are costly on conven-tional systems mainly due to inefficient access patterns, limited data reuse and roundtrip data traversal throughout the mem-ory hierarchy. This paper presents a two pronged approach for efficient data reorganization, which combines (i) a pro-posed DRAM-aware reshape accelerator integrated within 3D-stacked DRAM, and (ii) a mathematical framework that is used to represent and optimize the reorganization operations. We evaluate our proposed system through two major use cases. First, we demonstrate the reshape accelerator in per-forming a physical address remapping via data layout trans-form to utilize the internal parallelism/locality of the 3D-stacked DRAM structure more efficiently for general purpose workloads. Then, we focus on offloading and accelerating commonly used data reorganization routines selected from the Intel Math Kernel Library package. We evaluate the energy and performance benefits of our approach by comparing it against existing optimized implementations on state-of-the-art GPUs and CPUs. For the various test cases, in-memory data reorganization provides orders of magnitude performance and energy efficiency improvements via low overhead hardware. 1