Scaling Distributed Cache Hierarchies through Computation and Data Co-Scheduling

Abstract—Cache hierarchies are increasingly non-uniform, so for systems to scale efficiently, data must be close to the threads that use it. Moreover, cache capacity is limited and contended among threads, introducing complex capacity/latency tradeoffs. Prior NUCA schemes have focused on managing data to reduce access latency, but have ignored thread placement; and applying prior NUMA thread placement schemes to NUCA is inefficient, as capacity, not bandwidth, is the main constraint. We present CDCS, a technique to jointly place threads and data in multicores with distributed shared caches. We develop novel monitoring hardware that enables fine-grained space al-location on large caches, and data movement support to allow frequent full-chip reconfigurations. On a 64-core system, CDCS outperforms an S-NUCA LLC by 46 % on average (up to 76%) in weighted speedup and saves 36 % of system energy. CDCS also outperforms state-of-the-art NUCA schemes under different thread scheduling policies. Index Terms—cache, NUCA, thread scheduling, partitioning I