ASSESSMENT OF PERSON EMOTION BASED ON DIGITAL IMAGES

V tem magistrskem delu smo se ukvarjali s študijem postopkov za razpoznavanje človeških čustev na osnovi digitalnih posnetkov. V praktičnem delu te magistrske naloge smo razvili izviren računalniški razpoznavalni sistem, ki temelji na teksturnih značilkah. Algoritem v prvem koraku poišče grob položaj obraza na vhodni sliki, zatem pa v dobljeni regiji računamo ujemanje obraza z modelom AAM (Active Appearance Models). Nato rotiramo obraz za izračunan kot v pravilen frontalni položaj. V zadnji fazi algoritma s pomočjo koordinat iz modela AAM natančno izrežemo obraz iz slike, zatem pa obraz aproksimiramo z elipso, s čimer odstranimo odvečne informacije z slike. Obrezan obraz na koncu popišemo s teksturno značilko HOG (Histogram of Oriented Gradients). Vmesni rezultat je histogram, ki ga posredujemo v stroje SVM (Support Vector Machines) za klasifikacijo, pri čemer za vsako od šestih osnovnih emocij naučimo lasten SVM. Razvili smo dve varianti algoritma, in sicer algoritem na osnovi statičnih 2D slik in algoritem na osnovi slik razlik. Prvi algoritem uporablja za razpoznavanje čustev le trenutno sliko opazovane osebe, medtem ko drugi algoritem detektira spremembe obraza pri izražanju čustev. Algoritem na osnovi statičnih 2D slik smo nadgradili z algoritmom Adaboost, algoritem na osnovi slik razlik pa smo razširili s tremi variantami, kjer pri gradnji histograma s pomočjo interpolacije vnesemo še vmesne korake pri spreminjanju obraza iz nevtralnega v obraz z izkazanim čustvom. Razvite algoritme smo testirali na dveh javno dostopnih testnih podatkovnih bazah: bazi MMI Facial Expression Database (MMI) in bazi Cohn-Kanade. Z algoritmom na osnovi statičnih 2D slik smo na bazi MMI dosegli najvišjo uspešnost 76,31 %, na bazi CK pa 91,49 %. Z algoritmom na osnovi slik razlik pa smo na bazi MMI dosegli 74,63 % uspešnost, na bazi CK pa se je uspešnost prepoznavanja čustev povzpela kar na 95,64 %.In our work, we have been studying the procedures of human emotion recognition based on digital images. We have developed an original computer recognitional sistem, which is based on feature descriptor, in the practical part of our work. The first step is the algorithm defining the approximate position of the face in the input image, which is followed by calculations of matching of the face with the AAM model (Active Appearance Model) in the region. Then we rotate the face according to the calculated angle to its frontal position. The last step of the algorithm is cutting the face out of the image, using coordinates, and approximating it with an ellipse in order to remove the abundant information from the image, which is followed by using the feature descriptor HOG (Histogram of Oriented Gradients). The intermediate result is a histogram, which needs to be classified by SVM machines (Support Vector Machines), whereas one machine is taught one particular emotion. We have developed two versions of the algorithm. The first one is based on static 2D images, and the other is based on images of differences. The first one uses a current image of the observed person, whereas the second one detects changes of the face when expressing emotions. We have upgraded the algorithm based on static 2D images with algorithm Adaboost, and expanded the one based on images of differences with three versions, which include interpolating additional intermediate changes of the face from neutral to the one with the revealed emotion, to construction of the histograms. We have tested the algorithms on two public test databases: MMI Facial Expression Database (MMI) and Cohn-Kanade database. The 2D static images based algorithm\u27s highest score of effectiveness was 76.31% on MMI database and 91.49% on CK database. The images of differences based algorithm\u27s highest score of effectiveness was 74.63% on MMI and 95.64% on CK