Improving the Robustness of Neural Networks to Noisy Multi-Level Non-Volatile Memory-based Synapses

International audienceThe implementation of Artificial Neural Networks(ANNs) using analog Non-Volatile Memories (NVMs) for synapticweights storage promises improved energy-efficiency and higherdensity compared to fully-digital implementations. However,NVMs are prone to variability, resulting in a degradation of theaccuracy of ANNs. In this paper, a general methodology to evaluate and enhance the accuracy of neural networks implementedwith non-ideal multi-level NVMs is presented. A hardware faultmodel distinguishing two types of errors, namely static anddynamic, capturing the variability of NVMs is proposed. Considering various neural networks, it is shown that error-awaretraining highly increases the robustness to errors compared toa standard, error-agnostic, training. Moreover, Recurrent NNs(RNNs) and Spiking NNs (SNNs) are found to be inherentlymore robust to dynamic errors than Convolutional NNs (CNNs).In addition, new insights on the adaptability of neural networksto noisy multi-level NVMs are presented, which could furtherimprove their robustness in this context. The methodology aimsat providing tools for hardware-software co-design, paving theway for a broader use of multi-level NVM-based synapses