Power Profiling of Machine Learning Accelerators using MLPerf

There exists a vast amount of possibilities of hardware platforms and models when implementing a machine learning problem. In order to be able to choose the best approach, an estimator of important parameters like latency, power consumption and accuracy would be beneficial. To build such a tool, it is necessary to characterize hardware and models first. This is done in this work by reproducing the results of a study that examines the Google Coral USB Accelerator and the Intel Neural Compute Stick 2. Especially, the power consumption is looked at in more detail by using a sampling rate of 100kS/s. The findings include power profiles of the mentioned hardware accelerators executing MobileNetV1 and Inception V1 by means of the MLPerf Inference benchmark. Additionally, the accuracy and latency are examined and the results are compared to the related study. Moreover, a measurement setup, including a USB dongle for simple and reliable data acquisition, is presented. A sampling rate of 500kS/s allows it to evaluate each individual layer of a model regarding power and latency during inference. The presented work serves as a basis for further investigations in the field of parameter estimation