Active Routing: Compute on the Way for Near-Data Processing

The explosion of data availability and fast data analytic requirements led to the advent of data-intensive applications, characterized by their large memory footprint and low data reuse rate. These data intensive applications place a significant amount of stress on modern memory systems and communication infrastructure. These workloads, ranging from data analytics to machine learning, exhibit a considerable number of aggregation operations over large data sets, whose performance is limited by the memory stalls due to widening gap between high CPU compute density and deficient memory bandwidth. This work presents Active-Routing, an In-Network Compute Architecture enabling compute on the way for Near-Data Processing (NDP), which reduces data movements across the memory hierarchy. It moves computations close to data location onto the memory network switches which operate concurrently and construct an Active-Routing tree. The network is enabled with computing capability to optimize the aggregation operations on a dynamically built routing tree to reduce network traffic and parallelize computation across the memory network. Evaluations in this work show that Active-Routing can achieve up to 6x speedup with an average of 75% performance improvement across various benchmarks compared to a Baseline system integrated with a die-stacked memory network