Memory Management for Big Memory Systems

The memory system has been evolving at a fast pace recently, driven by the emergence of large-scale applications and the advance in hardware technology. This trend calls for the birth of big memory systems with extreme heterogeneity, which combines multiple memory technologies with different latency, bandwidth, and capacity to construct main memory.The conventional memory management methods are not adequate to handle the increasing complexity that heterogeneity brings forward, and will fail to deliver the full potential of the new memory. Specifically, the heterogeneity of memory systems brings a substantial disparity in the performance and efficiency, making the decision of which technology to use at what times intricate.The performance of heterogeneous memory-based big memory systems highly depends on the data locality.By dynamically profiling memory access behaviors, memory management solutions move frequently accessed (or hot) data from slow to fast memory and less frequently accessed (or cold) data from fast to slow memory. However, the decision of how to spread data across all memory components in big memory systems, which data to move, and at what times is not trivial. This dissertation proposes a series of techniques, spanning from architecture, runtime systems, operating systems, programming models, to applications and algorithms, which are used to efficiently identify which data in applications are performance-critical, timely determine when to perform data migration, and select where to place those data in big memory systems. Specifically, this dissertation contains six common scenarios using big memory systems with big memory applications such as machine learning/artificial intelligence applications, large-scale scientific simulation, and in-memory databases. We propose several software techniques for the efficient use of big memory systems in all scenarios. By doing so, this dissertation identifies bottlenecks in the existing memory management solution, explores the unique characteristics of different types of applications, and bridges gaps from different system stacks to remedy bottlenecks. Evaluation results based on real-world big memory systems illustrate that with efficient data placement, our solutions outperform existing OSes-based and hardware-based memory management solutions