Using Artificial Intelligence for Gameplay Testing On Turn-Based Games

Background. Game development is a constantly evolving multi billion dollar in-dustry, and the need for quality products is very high. Testing the games however isa very time consuming and tedious task, often coming down to repeating sequencesuntil a requirement has been met. But what if some parts of it could be automated,handled by an artificial intelligence that can play the game day and night, givingstatistics about the gameplay as well as reports about errors that occurred duringthe session? Objectives. This thesis is done in cooperation with Fall Damage Studio AB, andaims to find and implement a suitable artificial intelligent agent to perform auto-mated test on a game Fall Damage Studio AB are currently developing, ProjectFreedom. The objective is to identify potential problems, benefits, and use casesof using a technique such as this. A secondary objective is to also identify what isneeded by the game for this kind of technique to be useful. Methods. To test the technique, aMonte-Carlo Tree Searchalgorithm was identi-fied as the most suitable algorithm and implemented for use in two different typesof experiments. The first being to evaluate how varying limitations in terms of thenumber of iterations and depth affected the results of the algorithm. This was doneto see if it was possible to change these factors and find a point where acceptablelevels of plays were achieved and further increases to these factors gave limited en-hancements to this level but increased the time. The second experiment aimed toevaluate what useful data can be extracted from a game, both in terms of gameplayrelated data as well as error information from crashes. Project Freedom was onlyused for the second test due to constraints that was out of scope for this thesis totry and repair. Results. The thesis has identified several requirements that is needed for a game touse a technique such as this in an useful way. For Monte-Carlo Tree Search specifi-cally, the game is required to have a gamestate that is quick to create a copy of anda game simulation that can be run in a short time. The game must also be testedfor the depth and iteration point that hit the point where the value of increasingthese values diminish. More generally, the algorithm of choice must be a part of thedesign process and different games might require different kind of algorithms to use.Adding this type of algorithm at a late stage in development, as was done for thisthesis, might be possible if precautions are taken. Conclusions. This thesis shows that using artificial intelligence agents for game-play testing is definitely possible, but it needs to be considered in the early part ofthe development process as no one size fits all approach is likely to exist. Differentgames will have their own requirements, some potentially more general for that typeof game, and some will be unique for that specific game. Thus different algorithmswill work better on certain types of games compared to other ones, and they willneed to be tweaked to perform optimally on a specific game