Embracing Heterogeneity in Modern GPUs

Graphics Processing Units is one of the most widely adopted parallel computing engines for modern applications. However, due to the “memory wall”, the scaling of GPUs is lagging behind the ever-growing complexity of application algorithms and ever-increasing data volume of inputs. Near-data computing (NDC) is a widely-acknowledged computing paradigm that alleviates the memory wall problem by offloading computation to data instead of conventional fetching data to computation. While the NDC GPU architectures have been proposed and studied, there are several key questions that remain unanswered: i) what portions of an application's execution should be launched as GPU device kernels that can benefit from the executions on NDC GPU cores? ii) How to schedule these device kernels while considering the data reuse and sharing among them? iii) Where to execute these kernels if there are multiple potential candidate cores