Evaluating the quality of 3D character animation produced by artificial neural networks : A user study

Background. In recent years the use of Artifical Neural Networks(ANN) to generate character animations have expanded rapidly. However comparisons of the perceived realism and quality of these new approaches when compared to the results of a traditional pipeline have been lacking or non-existent. Objectives. This thesis aims to show initial data on whether the visual quality of one of these novel approaches is perceived to be of higher visual quality than similar keyframe-based approaches and if so why. As such the objectives of this thesis is to produce animations as a base of comparison for the method. Method. Keyframe animations performing similar actions to an ANN-based plug-in for the game engine unity were handcrafted after which a questionnaire study was performed. This study was sent to willing participants both experienced and inexperienced to gauge public opinion. Results. The results indicated that participants considered the ANN-based plug-in animations to overall be more realistic, natural, smooth and appealing. Statistical analysis via t-test shows a high statistical significance when comparing opinions on quality between the two sets. Conclusions. The ANN-based approach was considered by participants to be of superior visual quality due to reasons stated above. All experienced participants correctly guessed which of the two animation sets were AI-based and 1/3 of the inexperienced. However the inexperienced participants who guessed wrong stated similar motivations for their guess to those who guessed right. Which could imply an uncertainty about the capabilities of AI in the public consciousness that wasn’t accounted for in this thesis.