Comparison of Two Different Methods of Generating Physics-Based Character Animation using Reinforcement Learning

In the area of Physics-Based Animation using Reinforcement Learning (RL), multiple games and virtual simulations have had it as an objective throughout the years because of the added realism it brings to motions of characters. It also brings a realistic and practical interaction between the characters and their environment. By using a physically simulated ragdoll and Reinforcement Learning, the ragdoll can learn how to walk and balance. This has the potential of bringing realistic and interactive real-time animations, of normally rigid animations, to life. No longer do animators have to animate every single scenario of foot placement, collision, or fall because the physics-based character will produce realistic motions for every unique scenario. Previous work includes the article Data-Driven Responsive Control of Physics-Based Characters (2019)\cite{bergaminDReConDatadrivenResponsive2019} which uses a motion matching and Imitation method during its reinforcement learning to assure that the motion of the characters looks more human-like and realistic. The article, Emergence of Locomotion Behaviours in Rich Environments (2017)\cite{heessEmergenceLocomotionBehaviours2017} does not use motion imitation which means that the agent can develop any kind of motions, realistic-looking or not. None of these previous works, which uses different methods, have been tested against each other. The research question of the thesis is as follows: Which benefits does motion imitation bring to physics-based imitation? Both experiments and a survey were used to answer this question. To do this, the two separate methods of generating physics-based animation using Reinforcement Learning was implemented. One that used motion imitation, and one without using motion imitation. These were then compared against each other in terms of performance, responsiveness, realism, and appeal. The results of the experiments show that none of the versions had a better run-time performance, but the version without motion imitation did have a shorter learning time performance. This may be irrelevant though because the version without motion imitation took more attempts to develop, which resulted in additional time spent on learning. The version that used motion imitation had significantly better responsiveness which was measured by checking the different agent's ability to follow a certain goal speed. The survey was used to determine the realism and appeal of the different versions and the results show that all participants preferred the version that used motion imitation in terms of realism. The majority of the participants also thought the version with motion imitation had a more appealing motion