The Benefit of the Doubt: Uncertainty Aware Sensing for Edge Computing Platforms

Neural networks (NNs) have drastically improved the performance of mobile and embedded applications but lack measures of “relia- bility” estimation that would enable reasoning over their predic- tions. Despite the vital importance, especially in areas of human well-being and health, state-of-the-art uncertainty estimation tech- niques are computationally expensive when applied to resource- constrained devices. We propose an efficient framework for predic- tive uncertainty estimation in NNs deployed on edge computing platforms with no need for fine-tuning or re-training strategies. To meet the energy and latency requirements of these systems the framework is built from the ground up to provide predictive un- certainty based only on one forward pass and a negligible amount of additional matrix multiplications. Our aim is to enable already trained deep learning models to generate uncertainty estimates on resource-limited devices at inference time focusing on classifica- tion tasks. This framework is founded on theoretical developments casting dropout training as approximate inference in Bayesian NNs. Our novel layerwise distribution approximation to the convolu- tion layer cascades through the network, providing uncertainty estimates in one single run which ensures minimal overhead, espe- cially compared with uncertainty techniques that require multiple forwards passes and an equal linear rise in energy and latency re- quirements making them unsuitable in practice. We demonstrate that it yields better performance and flexibility over previous work based on multilayer perceptrons to obtain uncertainty estimates. Our evaluation with mobile applications datasets on Nvidia Jetson TX2 and Nano shows that our approach not only obtains robust and accurate uncertainty estimations but also outperforms state-of- the-art methods in terms of systems performance, reducing energy consumption (up to 28–folds), keeping the memory overhead at a minimum while still improving accuracy (up to 16%)