On-Chip Network Design Considerations for Compute Accelerators

There has been little work investigating the overall perfor-mance impact of on-chip communication in manycore com-pute accelerators. In this paper we evaluate performance of a GPU-like compute accelerator running CUDA work-loads and consisting of compute nodes, interconnection net-work and the graphics DRAM memory system using de-tailed cycle-level simulation. First, we study performance of a baseline architecture employing a scalable mesh network. We then propose several microarchitectural techniques to exploit the communication characteristics of these applica-tions while providing a cost-effective (i.e., low area) on-chip network. Instead of increasing costly bisection bandwidth, we increase the the number of injection ports at the mem-ory controller router nodes to increase terminal bandwidth at the few nodes. In addition, we propose a novel “checker-board ” on-chip network which alternates between conven-tional, full-routers and half-routers with limited connectiv-ity. This network is enabled by limited communication of the many-to-few traffic pattern. We describe a minimal routing algorithm for the checkerboard network that does not in-crease the hop count