Optimizing Locality-Aware Memory Management of Key-Value Caches

The in-memory cache system is a performance-critical layer in today's web server architectures. Memcached is one of the most effective, representative, and prevalent among such systems. An important problem is on its memory allocation. The default design does not make the best use of the memory. It is unable to adapt when the demand changes, a problem known as slab calcification. This paper introduces locality-aware memory allocation (LAMA), which addresses the problem by first analyzing the locality of Memcached's requests and then reassigning slabs to minimize the miss ratio or the average response time. By evaluating LAMA using various industry and academic workloads, the paper shows that LAMA outperforms existing techniques in the steady-state performance, the speed of convergence, and the ability to adapt to request pattern changes, and overcome slab calcification. The new solution is close to optimal, achieving over 98 percent of the theoretical potential. Furthermore, LAMA can also be adopted in resource partitioning to guarantee quality-of-service (QoS).National Science Foundation of China [61232008, 61272158, 61328201, 61472008, 61672053]; 863 Program of China [2015AA015305]; National Science Foundation [CCF-1629376, CNS-1319617, CNS1527076, CSR-1618384, CSR-1422342]; Huawei; IBM CAS Faculty Fellow programSCI(E)ARTICLE5862-8756