Power-optimal mapping of CNN applications to cloud-based Multi-FPGA platforms

Multi-FPGA platforms like Amazon Web Services F1 are perfect to accelerate multi-kernel pipelined applications, like Convolutional Neural Networks (CNNs). To reduce energy consumption, we propose to upload at runtime the best power-optimized CNN implementation for a given throughput constraint. Our design method gives the best number of parallel instances of each kernel, their allocation to the FPGAs, the number of powered-on FPGAs and their clock frequency. This is obtained by solving a mixed-integer, non-linear optimization problem that models power and performance of each component, as well as the duration of the computation phases-data transfer between a host CPU and the FPGA memory (typically DDR), data transfer between DDR and FPGA, and FPGA computation. The results show that the power saved compared to simply clock gating the fastest implementation is obviously very high, but it is also much more significant than simply scaling the frequency of the fastest implementation or replicating the slowest implementation on multiple FPGAs.This work was partially supported in part by the European Commission through the ECOSCALE Project under Grant H2020-ICT671632, in part by the Spanish Ministry for Economy and Competitiveness and the European Union (FEDER Funds) under Grant TIN2017-86727-C2-1- R and Grant FPI 2015, and in part by the Generalitat de Catalunya under Grant 2017 SGR 786.Peer ReviewedPostprint (author's final draft