Asynchronous Distributed ADMM for Consensus Optimization

Distributed optimization algorithms are highly attractive for solving big data problems. In par-ticular, many machine learning problems can be formulated as the global consensus opti-mization problem, which can then be solved in a distributed manner by the alternating direc-tion method of multipliers (ADMM) algorithm. However, this suffers from the straggler problem as its updates have to be synchronized. In this paper, we propose an asynchronous ADMM al-gorithm by using two conditions to control the asynchrony: partial barrier and bounded delay. The proposed algorithm has a simple structure and good convergence guarantees (its conver-gence rate can be reduced to that of its syn-chronous counterpart). Experiments on different distributed ADMM applications show that asyn-chrony reduces the time on network waiting, and achieves faster convergence than its synchronous counterpart in terms of the wall clock time. 1