COLLABORATIVE EXECUTION OF NEURAL NETWORKS ON MODERN EMBEDDED SYSTEMS

In recent years, machine learning applications are progressing on mobile systems for enhanced user experience. Unlike the traditional approach where training and inference both were executed on the cloud, now, for the security and privacy concerns, the applications demand to be performed entirely on the edge device itself. With mobile systems embedded more with heterogeneous devices such as multi-core CPU, GPU, and other neural accelerators, the inferences are performed efficiently and with high accuracy. On increasing demand of running applications on these devices, it is important to execute inference workloads among processing elements to ensure high throughput, which becomes challenging. In this work, we study the characteristics exhibited by the neural networks across the processing elements and explore executing inference workloads efficiently on these devices considering metrics such as inference time and frame rate