Efficient Memory Virtualization Reducing Dimensionality of Nested Page Walks

Abstract — Virtualization provides value for many workloads, but its cost rises for workloads with poor memory access locali-ty. This overhead comes from translation lookaside buffer (TLB) misses where the hardware performs a 2D page walk (up to 24 memory references on x86-64) rather than a native TLB miss (up to only 4 memory references). The first dimen-sion translates guest virtual addresses to guest physical ad-dresses, while the second translates guest physical addresses to host physical addresses. This paper proposes new hardware using direct segments with three new virtualized modes of op-eration that significantly speed-up virtualized address transla-tion. Further, this paper proposes two novel techniques to ad-dress important limitations of original direct segments. First, self-ballooning reduces fragmentation in physical memory, and addresses the architectural input/output (I/O) gap in x86-64. Second, an escape filter provides alternate translations for exceptional pages within a direct segment (e.g., physical pages with permanent hard faults). We emulate the proposed hardware and prototype the software in Linux with KVM on x86-64. One mode—VMM Direct—reduces address translation overhead to near-native without guest application or OS changes (2 % slower than na-tive on average), while a more aggressive mode—Dual Direct— on big-memory workloads performs better-than-native with near-zero translation overhead. Keywords-virtual memory; virtual machines; virtualization; translation lookaside buffer. I