Approximate multipliers in neural network training

V magistrski nalogi realiziramo nevronsko mrežo večslojni perceptron in štiri različne učne algoritme. Nevronska mreža in učni algoritmi so modularni kar pomeni, da lahko preminjamo format realnih števil s katerimi računajo. Spreminjamo lahko med plavajočo vejico in fiksno vejico. Pri fiksni vejici nastavimo širino operandov, določimo položaj binarne vejice in izberemo množilnik s katerim izračunamo vse produkte. S fiksno vejico in s približnimi množilniki simuliramo učenje nevronskih mrež na manj zmogljivi strojni opremi. Pri fiksni vejici so vse aritmetične operacije realizirane tako, da so enostavno izvedljive s strojno opremo. Med seboj primerjamo delovanje različnih učnih algoritmov, pri različnih formatih in pri uporabi različnih množilnikov. Delovanje primerjamo s tremi različnimi nabori podatkov, od tega sta dva nabora iz zbirke Proben1 [1], tretji pa je nabor MNIST [2]. Primerjava je pokazala, da je najbolj primeren učni algoritem, za učenje nevronskih mrež na manj zmogljivi strojni opremi, metoda najstrmejšega sestopa.In the master’s thesis, we realize a neural network multilayer perceptron and four different learning algorithms. The neural network and learning algorithms are modular and allow to change the number format for operand representation. We can choose between floating-point and fixed-point formats. At a fixed point, we additionally set the width of operands, determine the position of the binary point and select the multiplier with which we compute the products. With fixed point and approximate multipliers, we simulate neural network training on less powerful hardware. We designed all arithmetic operations in a fixed point to be easily realizable in hardware. We compare neural network training with different learning algorithms in different number formats with different multipliers. We run experiments on three datasets, of which two are from the Proben1 [1] collection, and the third is the MNIST [2] dataset. The comparison shows that the most suitable learning algorithm for training neural networks on less powerful hardware is the method of steepest descent