Hardware Approximation of Exponential Decay for Spiking Neural Networks

Spiking neural networks (SNNs) may enable low-power intelligence on the edge by combining the merits ofdeep learning with the computational paradigms found in thehuman neo-cortex. The choice of neuron model is an openresearch topic. Many spiking models implement neural dynam-ics from biology that involve one or more exponential decayfunctions. Previous work focused on accurate modeling of theexponential decay function on neuromorphic hardware to thelast significant bit (LSB). In this paper, we explore the limitsof error resilience in SNNs by aggressively approximating theirexponential decay functions and allowing for losses within ourbit precision. Three approximation methods are presented andimplemented with varying degrees of precision resulting in 10different implementations. Their hardware cost and inferenceaccuracy on benchmark networks and applications are compared.To improve the inference accuracy, we implemented fine-tuning.We also introduced hardware programmability for certain timeconstants as hyperparameters. Our results show resilience to lossyapproximation, fast fine-tuning, and a low energy consumptionof 47 fJ per operation