Criticality-Aware Memory Systems

Research on computer memory systems has been of increasing importance over the last decade, as they have become a significant bottleneck for application performance. While newer memory systems offer increased memory level parallelism, they cannot be used blindly due to contention for shared resources, making a beneficial and valid sequencing of memory requests requisite in order to exploit these improvements. Traditional approaches to improving this sequencing rely on highly sophisticated memory systems, where significant amounts of inference are often required to make these sophisticated decisions. Unfortunately, this design philosophy may no longer be sustainable. For example, as memory clock frequencies continue to scale while processor frequencies remain stagnant, sophisticated memory controllers are already being squeezed out, forcing computer architects to revert to simpler designs. We use this as an opportunity to symbiotically involve the processor cores in the decision-making process, simultaneously offloading the complexity from these memory decision makers while extracting richer information on each memory operation. This work studies the concept of load criticality, where the processors themselves identify the loads which they believe to be most important. Using loads that block at the end of the processor pipeline as an indicator of criticality, we annotate these load block predictions onto memory requests, for use by various components in memory. Our research finds that even using small, sim- ple predictors for load criticality can offer comparable performance to complex state-of-the-art schedulers for both parallel applications and multiprogrammed workloads on a contemporary multicore system. This same predictor can be used to obtain significant performance improvements and energy savings when using hardware prefetchers. Ultimately, our criticality-aware design approach achieves the performance of traditionally-complex memory systems, and does so with trivial overheads that are attractive for future commercial adoption