Neural network training methods based on genetic algorithms

Učenje neuronskih mreža spada u probleme optimizacije. Višeslojne unaprijedne statičke neuronske mreže uspješno se uče algoritmom povratnog rasprostiranja pogreške. Genetski algoritmi također se koriste u problemima optimiranja. U ovom radu je razvijen genetski algoritam za učenje općenite neuronske mreže. Njegova uspješnost procjenjuje se sporedbom sa učenjem povratnim rasprostiranjem pogreške. Također se prikazuje hibridna metoda – kombinacija obiju metoda učenja, te procjenjuje njezina kvaliteta. Prikazuje se i genetski algoritam za učenje neuronske mreže na debalansiranim skupovima podataka za učenje. U tu svrhu implementirani su povratno rasprostiranje pogreške, genetski i hibridni algoritam. Genetski algoritam testiran je za nekoliko stupnjeva slobode: zapis kromosoma, veličinu populacije, vjerojatnost mutacije i način primjene mutacije. Algoritmi se testiraju na neuronskoj mreži za razvrstavanje uzoraka u dvije klase. Rezultati ispitivanja ukazuju da se genetski algoritam može uspješno koristiti kao metoda za učenje neuronskih mreža. Genetski algoritam sa zapisom kromosoma binarnim nizom dao je osjetno bolje rezultate nego povratno rasprostiranje pogreške, a genetski algoritam sa zapisom brojevima pomičnom točkom nešto lošije. Kombinacija metoda nije pokazala nikakav boljitak u odnosu na genetski algoritam primjenjen zasebno. U roblemima sa debalansiranim skupovima podataka, genetski algoritam je jednostavniji za implementaciju nego odgovarajući algoritam povratnog rasprostiranja pogreške. I za takve probleme genetski algoritam daje dobre rezultate.Neural network learning is an optimisation problem. Multilayered, feedforward static neural networks are well trained by using error-backpropagation algorithm. Genetic algorithms are also used for the optimisation problem solving. In this master thesis a genetic algorithm for neural network training is developed. Its benefit in this field is mesaured by comparing it to error back-propagation algorithm. The hybrid method – combination of both the error back-propagation and genethic algorithm is also shown, and its quality is evaluated. And finally, modified genetic algorithm for imbalanced learning data set is presented and tested. Genetic, error back-propagation and hybrid algorithms were implemented. Genetic algorithm is tested for different chromosome representations, population size, mutation probability and the way mutation is applied. All implemented learning algorithms are tested on neural network for pattern classification in 2 classes. Testing results indicate that the genetic algorithm can be used successfully for the neural network training. Training results with genetic algorithm with floating-point chromosome representation were much better than results achieved with error back propagation. At the same time, training results achieved with genetic algorithm with floating point chromosome representation were slightly worse. Hybrid method did not show any progress when compared to training with the genetic algorithm only. Genetic learnnig for imbalanced learning data sets is easier to implement than an adequate error back-propagation. In these cases training with the genethic algorithm was also successful