Self-Organizing Agents for Reinforcement Learning in Virtual Worlds

Abstract — We present a self-organizing neural model for creating intelligent learning agents in virtual worlds. As agents in a virtual world roam, interact and socialize with users and other agents as in real world without explicit goals and teachers, learning in virtual world presents many challenges not found in typical machine learning benchmarks. In this paper, we highlight the unique issues and challenges of building learning agents in virtual world using reinforcement learning. Specif-ically, a self-organizing neural model, named TD-FALCON (Temporal Difference- Fusion Architecture for Learning and Cognition), is deployed, which enables an autonomous agent to adapt and function in a dynamic environment with im-mediate as well as delayed evaluative feedback signals. We have implemented and evaluated TD-FALCON agents as virtual tour guides in a virtual world environment. Our experimental results show that the agents are able to adapt and improve their performance in real time. To the best of our knowledge, this is one of the few in-depth works on building complete learning agents that adapt their behaviors through real time reinforcement learning in virtual world. I