Q-learning and Sarsa Agents in Bargaining Game

This paper focuses on sensitivity of learning mechanisms applied to agents in agent-based simulation and explores criteria for employing such learning mechanisms by comparing simulation results derived from agents who have di#erent learning mechanisms. Specifically, we employ two types of reinforcement learning in this study, Q-learning and Sarsa. Through an analysis of simulation results in a bargaining game as one of the fundamental examples in game theory, the following implications have been revealed: (1) results between Q-learning and Sarsa agents are mostly the same from one viewpoint, but di#er from another viewpoint; (2) Sarsa agents are superior to Q-learning agents in negotiation, while Sarsa agents cannot acquire rational behaviors, which can be acquired by Q-learning agents; and (3) implications (1) and (2) derive the following criteria that provide a rough guideline for employing learning mechanisms in agent-based simulation: (3-a) a discount factor in learning mechanisms and (3-b) comparison of simulation results mixing di#erent learning agents