Distributed artificial neural network architecture over wireless sensor network

The objective of this thesis is to study how to distribute an Artificial Neural Network (ANN) over a Wireless Sensor Network (WSN). On the one hand, ANNs are Artificial Intelligent (AI) algorithms inspired on our brains that exhibit a great amount of parallelism. The main advantage of the ANNs is that they can learn non-linear features and patterns from examples. On the other hand, WSNs are a technology with increasing interest due to the rise of the IoT (Internet of the Things). Nodes of a WSN usually are unattended and powered by batteries in remote or difficult-toaccess places. Hence, their resources are limited in order to maximize their lifetime. However, more intelligent WSNs are demanded, so, to merge these two kinds of networks seems to be natural. This thesis is about how to do it in an efficient way. In order to achieve a fast and energy-efficient implementation of the ANNs and due to the fact that they exhibit a great amount of parallelism, implementation into FPGAs was chosen. Microcontrollers and DSP processors may be too slow, and GPUs consume too much energy. An FPGA is a flexible intermediate option. Although there is already a lot of research about hardware implementation of ANNs into FPGAs, to the best of the authors’ knowledge at the beginning of this thesis there was not any freely available IP (Intellectual Property). Thus, we opt for designing our own IP, making it highly configurable to allow its use in a great variety of applications. The methodology used on this thesis was the following: First, to choose an application example in order to be able to get measurable data on simulations and implementations. The initial application example was to compress images using auto-encoders. It was chosen because authors had previous works on low-power compressors for WSN to compare with. Auto-encoders are ANN architectures with as many outputs as inputs and a narrow hidden layer (i.e. with less neurons than the output layer). When an auto-encoder is trained though a supervised training algorithm the inputs are taken as the desired outputs, forcing the ANN to learn a compress representation of the inputs on the narrow hidden layer. However, due to finding a way to merge images from multiple sensors into a combined one to be learned, is difficult, and as a worthy real application was not found to justify it, this application was discarded. The second example application was car and pedestrian detection with low-cost radars. This application was chosen to compare results with the thesis of another colleague from the lab. However, the amount of collected examples was not enough to properly train the ANNs. Hence, finally the popular MNIST dataset of handwritten digit classification was chosen. In order to adapt the MNIST dataset to a multi-sensor scenario, the images was divided into 4 quadrants and each quadrant assigned to a different node of the WSN. This emulates 4 sensors catching partial information or different points of view of the same event. Second, to adjust and optimize the algorithms and the intended designs through high-level of abstraction simulations. Matlab scripts were used for optimization and off-line training of the hardware implementations of the ANNs. And SystemC programs to simulate the WSNs. Then, to design, verify and implement the ANN on the FPGA. Two methodologies were used for this step: the classical RTL (Register Transfer Level) using VHDL and though HLS (High-Level Synthesis) using SystemC. Therefore two IPs were obtained: one in VHDL and another in SystemC. As a result form designing and implementing the ANNs through two methodologies, an analysis about the configurability of both methodologies was extracted. The relevance of this analysis is, that it was found that, whereas high-level descriptions are more likely to be re-used, SystemC parameterization mechanisms are less robust and flexible than VHDL ones. A good parameterization is necessary to facilitate code re-use. The parameterization mechanisms of VHDL are the generics, and the SystemC ones are macros, templates and constructor arguments. Macros are limited to scalar values, constructor arguments are not synthesizables and templates have some limitations specified on the analysis. The challenge when parameterizeing an ANN is to configure separately a configurable number of modules. In this case the layers need to be separately configured (number of neurons, activation function, etc.), and the number of layers should be a parameter too. On SystemC is easily doable a separate configuration of a fixed number of modules or a configurable insertion of exactly identical (i.e. with the same configuration) modules, but not both at the same time. Complex macros have been used to solve this problem. Whereas in VHDL, using generic arrays on for-generate statements can easily solve the challenge. Key points and hints for improvement of future HLS tools have been given. Finally, the main result of this thesis is the developed framework for distributing ANNs over WSNs. This framework consists of the Matlab scripts to optimize and train the entire ANN needed, the SystemC programs to estimate the computation vs. communication energy consumptions of the WSN with the ANN distributed in order to balance it, and the IPs to implement the ANN fragments on the nodes with FPGAs