Reinforcement Learning in Multi-Party Trading Dialog

In this paper, we apply reinforcement learning (RL) to a multi-party trading sce-nario where the dialog system (learner) trades with one, two, or three other agents. We experiment with different RL algo-rithms and reward functions. The nego-tiation strategy of the learner is learned through simulated dialog with trader sim-ulators. In our experiments, we evaluate how the performance of the learner varies depending on the RL algorithm used and the number of traders. Our results show that (1) even in simple multi-party trad-ing dialog tasks, learning an effective ne-gotiation policy is a very hard problem; and (2) the use of neural fitted Q itera-tion combined with an incremental reward function produces negotiation policies as effective or even better than the policies of two strong hand-crafted baselines.