Efficient Keyword Spotting through Hardware-Aware Conditional Execution of Deep Neural Networks

Keyword spotting is a task that requires ultra-low power due to its always-on operation. State-of-the-art approaches achieve this by drastically pruning model size, yet often at the expense of accuracy. This work tackles this fundamental conflict between operating efficiency and accuracy in three ways: 1.) Exploiting dynamic neural network cascades for keyword spotting using an end-to-end hardware-aware training; 2.) Deriving the optimal number of stages and stage dimensions in function of the input class distributions; 3.) Using the low-latency response of the first stage for speculative execution of the later stages, training the dynamic cascade through a hardware-aware cost function. Results show the framework can generate cascade models optimized in function of the class distribution (background noise, target keywords and other-speech), reducing computational cost by 87% for always-on operation while maintaining the baseline accuracy of the most complex model of the cascade. On top of this, the hardware-aware speculative execution provides an additional 2x energy savings over the non-speculative case.status: accepte