DNN-Dataflow- Hardware Co-Design for Enabling Pervasive General-Purpose AI

Presented on November 2, 2018 at 8:45 a.m. in the Klaus Advanced Computing Building, Room 1116 East/West, Georgia Institute of Technology (Georgia Tech).Second Annual Center for Research into Novel Computing Hierarchies (CRNCH) Summit, November 2, 2018 at Georgia Tech.Tushar Krishna is an Assistant Professor in the School of Electrical and Computer Engineering at Georgia Tech. He has a Ph.D. in Electrical Engineering and Computer Science from MIT (2014), a M.S.E in Electrical Engineering from Princeton University (2009), and a B. Tech in Electrical Engineering from the Indian Institute of Technology (liT) Delhi (2007). Before joining Georgia Tech in 2015, Dr. Krishna spent a year as a post-doctoral researcher at Intel, Massachusetts. Dr. Krishna's research spans computer architecture, interconnection networks, networks-on-chip (NoC) and deep learning accelerators - with a focus on optimizing data movement in modern computing systems. He has 38 publications in leading conferences and journals, which have amassed over 4300 citations to date. Two of his papers got selected for IEEE Micro's Top Picks from Computer Architecture and two have won best paper awards. He coauthored a textbook on NoCs in 2017, and developed and maintains the Garnet2.0 NoC simulator within gem5 (www.gem5.org) that is being used by over a 100 researchers world-wide. He received the National Science Foundation (NSF) CRII award in 2018.Runtime: 32:52 minutesThe development of supervised learning based DL solutions today is mostly open loop. A typical DL model is created by hand-tuning the neural network (NN) topology by a team of experts over multiple iterations, often by trial and error, and then trained over gargantuan amounts of labeled data over weeks at a time to obtain a set of weights. The trained model hence obtained is then deployed in the cloud or at the edge over inference accelerators (such as GPUs, FPGAs, or ASICs). This form of DL breaks in the absence of labelled data, and/or if the model for the task at hand is unknown, and/or if the problem keeps changing. An AI system for continuous learning needs to have the ability to constantly interact with the environment and add and remove connections within the NN autonomously, just like our brains do. In this talk, we will briefly present our research efforts towards enabling general-purpose AI. First, we will present GeneSys, a HW-SW prototype of an Evolutionary Algorithm (EA)-based learning system, that comprises of a closed loop learning engine called EvE and an inference engine called ADAM. EvE is a genetic algorithm accelerator that can "evolve" the topology and weights of NNs completely in hardware for the task at hand, without requiring hand-optimization or back propogation training. ADAM continuously interacts with the environment and is optimized for efficiently running the irregular NNs generated by EvE, which today's suite of DL accelerators and GPUs are not optimized to handle. Next, we focus on the challenge of mapping a DNN model (developed via supervised or EA-based methods) efficiently over an accelerator (ASIC/GPU/FPGA). DNNs are essentially multi-dimensional loops, with millions of parameters and billions of computations. They can be partitioned in myriad ways to map over the compute array. Each unique mapping, or "dataflow" provides different trade-offs in terms of throughput and energy-efficiency, as it determines overall utilization and data reuse. Moreover, the right dataflow for a DNN depends heavily on the layer type, input activation to weight ratio, the accelerator microarchitecture, and its memory hierarchy. We will present an analytical tool called MAESTRO that we have been developing in collaboration with NVIDIA for formally characterizing the performance and energy-impact of dataflows in DNNs today. MAESTRO can be used at design-time, for providing quick first-order metrics at design-time when hardware resources (buffers and interconnects) are being allocated on-chip, and compile-time when different layers need to be optimally mapped for high utilization and energy-efficiency. Finally, we will present the micro-architecture of an open-source DNN accelerator called MAERI that is equipped to adaptively change the dataflow depending on the DNN layer currently being mapped by levering a runtime reconfigurable interconnection fabric