Border Pairs Method for the Artificial Neural Network learning

Disertacija opisuje novo metodo strojnega učenja - metodo mejnih parov. Opisana metoda je namenjena učenju večslojnega perceptrona, nevronske mreže s povezavami naprej, ki služi za prepoznavanje oziroma razvrščanje v razrede. Uvodoma je opisana problematika strojnega učenja, nevronska mreža večslojni perceptron (MLP) in njena klasična učna metoda backpropagation s poudarkom na njenih slabostih. V jedru disertacije najprej analiziramo lastnosti naučenega MLP. Pri tem se osredotočimo na učne vzorce v bližini meje in definiramo pojem mejnega para. Sledi analiza lastnosti mejnih parov, ki je podlaga za novo metodo razšumljanja, za novo metodo rojenja, in novo metodo konstruktivnega učenja. To učenje je lahko statično (offline), inkrementalno, dinamično (online) in s postopnim pozabljanjem starih učnih podatkov. Razšumljanje, rojenje in učenje smo testirali s simulacijo na računalniku. V ta namen smo uporabljali uveljavljene nabore učnih podatkov (zaradi primerljivosti rezultatov), realne nabore učnih podatkov (kot dokaz uporabnosti), kot tudi umetne (zaradi prilagajanja učnih podatkov našim potrebam). Primerjalna analiza je pokazala, da ima metoda mejnih parov nekaj dobrih lastnosti. Z njo smo uspešno razšumljali in rojili podatke, iskali značilke ter razvrščali podatke. Rezultati raziskav kažejo, da je obravnavana metoda zanesljiva, natančna, konstruktivna in odporna na šum in prekomerno učenje.Thesis describes a new method of machine learning - Border Pairs Method. The method described is intended for teaching Multi-Layer Perceptron - Feed-Forwarded Artificial Neural Network which is used for recognition or classification. Initially the problems of machine learning are described, as well as a description of the Multi-Layer Perceptron Neural Network and a description of Backpropagation, of its classical teaching method with an emphasis on its weaknesses. Firstly, in the core of the thesis we analyze the properties of trained MLP. In doing so, we focus on the learning patterns near the border and we define the concept of the Border Pair. Furthermore, the analysis of the properties of the border pairs, which is the basis of a new method of the noise reduction, of the clustering and a new method of constructive learning follow. This learning can be offline, incremental, online and with gradually forgetting the old learning patterns (unlearning). Noise cancellation, clustering and learning have been tested by simulation on a computer. Towards the end, we used the established sets of learning data (thanks to the comparability of results), real learning data sets (as a proof of the usability) as well as synthetic ones (due to the adaptation of learning data to our needs). The comparative analysis showed that the Border Pairs Method has some good characteristics. With it we have successfully reduced the noise, clustered the data, searched the features and classify the data. Results show that the researched method is reliable, accurate, constructive and insensitive to noise and overfitting