A methodology for the assessment of the behavior and performance of artificial agents

Computer games are becoming more popular for both entertainment and educational applications. The growth of this technology and its realm of use creates a new demand for artificial intelligence (AI) systems: as AI becomes more prevalent, it becomes crucial for it to have a natural, human feel to it in order to best support its application. Consequently, the need for a reliable means of testing and comparing the behavioral development of the artificial intelligence used within game applications becomes important. Turing’s test has been the staple in evaluating the “intelligence” of artificial agents in applications ranging from testing chatterbots to stopping web abuse. It is currently being used in evaluating the performance of specific artificial agents in particular games. In the following thesis, a methodology has been developed to provide a new contribution to the field of AI assessment. After bringing the perceptions of the human and AI onto the same level, the Turing test is used to evaluate the “humanness” of diverse agents in generalized environments. Results of a pilot study probing the validity of this methodology are presented.Masters of Computer EngineeringMastersUniversity of New Mexico. Dept. of Electrical and Computer EngineeringCaudell, ThomasCaudell, ThomasAngel, EdwardHeileman, Gregor