An Efficient Hardware Architecture for Exploiting Sparsity in Neural Networks

Sparsity – the presence of many zero values – is a pervasive property of modern deep neural networks, as it is inherently induced by state-of-the-art algorithmic optimizations. Recent efforts in hardware design for acceleration of neural networks have targeted the structure of computation of these workloads. However, when run on these value-agnostic accelerators, value sparsity is not exploited to provide performance or efficiency benefits, and instead results in wasted computation. In this thesis, we present architectural optimizations that efficiently leverage value sparsity in network weights in order to achieve significant performance benefits, with minimal hardware overhead. The culmination of this work is a hardware front-end (data fetching and staging unit) which, when paired with our novel, co-designed software scheduling algorithm, achieves more than a 2× speedup on average for the networks studied, with just an 8.2% overhead in compute area.M.A.S