On the agenda(s) of research on multi-agent learning

We survey the recent work in AI on multi-agent reinforcement learning (that is, learning in stochastic games). After tracing a representative sample of the recent literature, we argue that, while exciting, much of this work suffers from a fundamental lack of clarity about the problem or problems being addressed. We then propose five well-defined problems in multi-agent reinforcement learning and single out one that in our view is both well-suited for AI and has not yet been adequately addressed. We conclude with some remarks about how we believe progress is to be made on this problem