Resource Characterisation of Personal-Scale Sensing Models on Edge Accelerators

Edge accelerator is a class of brand-new purpose-built System On a Chip (SoC) for running deep learning models efficiently on edge devices. These accelerators offer various benefits such as ultra-low latency, sensitive data protection, and high availability due to their locality and are opening up interminable opportunities for building sensory systems in the real world. Naturally, in the context of sensory awareness systems, e.g., IoT, wearables, and other sensory devices, the emergence of edge accelerators is pushing us to rethink how we design these systems at a personal-scale. To this end, in this paper we take a closer look at the performance of a set of edge accelerators in running a collection of personal-scale sensory deep learning models. We benchmark eight different models with varying architectures and tasks (i.e., motion, audio, and vision) across seven platform configurations with three different accelerators including Google Coral, NVidia Jetson Nano, and Intel Neural Compute Stick. We report on their execution performance concerning latency, memory, and power consumption while discussing their current workflows and limitations. The results and insights lay an empirical foundation for the development of sensory systems on edge accelerators