A distributed actorcritic algorithm and applications to mobile sensor network coordination problems

Abstract—We introduce and establish the convergence of a distributed actor-critic method that orchestrates the coordination of multiple agents solving a general class of a Markov decision problem. The method leverages the centralized single-agent actor-critic algorithm of [1] and uses a consensus-like algorithm for updating agents ’ policy parameters. As an application and to validate our approach we consider a reward collection problem as an instance of a multi-agent coordination problem in a partially known environment and subject to dynamical changes and communication constraints. Index Terms—Markov decision processes, actor-critic methods, consensus, sensor networks, multi-agent coordination. I