Hardware acceleration of convolutional neural networks in embedded systems

Protekla decenija je donela izuzetan razvoj veštačke inteligencije i njenih podoblasti, posebno mašinskog učenja. Značajno poboljšanje rezultata dovelo je do rasta interesovanja ne samo u okviru naučne zajednice već i elektronske industrije. Šta više, algoritmi mašinskog učenja, su našli široku primenu i izvan navedenih okvira. Iako je celokupna podoblast doživela snažan razvoj i primenu u raznim sferama, jedna od najznačajnijih tiče se obrade slika (klasifikacija slika po raznim kriterijumima, lokalizacija objekata, segmentacija i slično). U ovakvim zadacima najviše su se istakle konvolucione neuronske mreže (klasa veštačkih neuronskih mreža). Jedna od mana ovog tipa veštačkih neuronskih mreža ogleda se u količini operacija koje je potrebno izvršiti da bi se slika procesirala. Prvobitno je ova kompleksnost ograničavala njihovu upotrebu na snažne računarske sisteme. Rast interesovanja za primenu dostignuća u raznim aplikacijama, povukla je razvoj i specijalizovanih akceleratora za kompaktne elektronske sisteme. Kako bi se prevazišla ograničenja resursa kakva vladaju u ovakvim sistemima, potrebno je razviti arhitekturu koja efikasno koristi raspoložive resurse. Tema ove doktorske disertacije je razvoj baš ovakvog akceleratora, namenjenog procesiranju konvolucionih neuronskih mreža. Pošto je reč o kompaktnom akceleratoru, konvolucione neuronske mreže nisu procesirane u izvornom obliku već su orezane kako bi se polazna kompleksnost redukovala, što je uobičajen pristup u ovakvim sistemima. Analizom prethodnih rešenja može se zaključiti da je razvoj algoritma za orezivanje često bio nezavisan od razvoja arhitekture što može ograničiti krajnje rezultate akceleratora. Ovakav ishod je najčešće posledica činjenice da algoritam za orezivanje nije razvijen imajući na umu raspoložive hardverske resurse koji će biti korišćeni prilikom implementacije akceleratora. Da bi se navedeno prevazišlo, postupak razvoja algoritma za orezivanje je u mnogome uključio karakteristike krajnjeg ciljanog sistema, a to je u slučaju ove disertacije programabilni hardver (FPGA, engl. Field-programmable-gate-array). Ostvareni rezultati idu u prilog hipotezi da je paralelni razvoj specijalizovanog algoritma za orezivanje i akceleratora od velike važnosti za postizanje visoke efikasnosti procesiranja konvolucionih neuronskih mreža u kompaktnim FPGA sistemima.The past decade has shown an extraordinary development of artificial intelligence and its subfields, especially machine learning. Significant improvement in results has led to an increase in interest not only within the scientific community but also in the electronics industry. Moreover, machine learning algorithms have found wide application beyond these limits. Although the entire subfield has experienced strong development and wide usage in various applications, one of the most used application concerns computer vision (image classification, object localization, segmentation, etc.). Convolutional neural networks (a class of artificial neural networks) has shown the best results among all other algorithms in such tasks. Disadvantage of this type of artificial neural network is reflected in the amount of operations that need to be performed in order to process the image. Initially, this complexity limited their use only to powerful computer systems. However, the growth of interest has led the development of specialized accelerators targeting also compact embedded systems. In order to overcome the resource constraints of such systems, it is necessary to develop an architecture that efficiently uses available resources. The topic of this doctoral thesis is the development of compact accelerator that can efficiently process convolutional neural networks. Since the developed acceelrator is intended for edge applications, convolutional neural networks are not processed in their original form. They are pruned in order to reduce the initial complexity, which is a common approach when targeting embedded systems. An analysis of previous solutions can conclude that the development of the pruning algorithm was often done independent of the development of the architecture which may limit the end results of the accelerator. This outcome is most often due to the fact that the pruning algorithm was not developed keeping in mind the available hardware. To overcome this, the created pruning algorithm takes into account the characteristics of the target system, which in the case is field-programmable-gate-array system on chip (FPGA SoC). Therefore, pruning algorithm is carefully tailored for FPGAs considering their resource characteristics. The achieved results support the hypothesis that the parallel development of a specialized pruning algorithm and accelerator is of great importance for achieving high processing efficiency of convolutional neural networks which are going to be deployed to the FPGA based systems