Multi-agent deep reinforcement learning based incentive mechanism for multi-task federated edge learning

Federated edge learning (FEL) is capable of training large-scale machine learning models without exposing the raw data of edge devices (EDs). Considering that the learning performance heavily depends on the active participation of EDs, it is essential to motivate the resource-limited EDs to contribute their efforts to learning tasks. In this paper, a learning-based multi-task FEL mechanism is proposed to design the economic incentive and participation contribution strategy jointly. Specifically, the incentive-based interaction between the edge servers and EDs is formulated as a multi-leader multi-follower Stackelberg game. Then, the theoretical analysis is provided to prove the existence and uniqueness of the Stackelberg equilibrium. To obtain the equilibrium solution under the incomplete information, a Markov decision process is formulated for the two-stage Stackelberg game. Considering the high dimensionality of the continuous action space, a multi-agent double actors deep deterministic policy gradient algorithm is employed to achieve the optimal training-ratio of EDs and the payment policies of edge servers. Numerical results validate the effectiveness and efficiency of our proposed incentive mechanism.This work was supported by the Key Research and Development Plan of Hubei Province, China (No. 2021BGD013); the Knowledge Innovation Special Project of Wuhan Science and Technology Bureau, China (No. 2022010801010255); the National Key R&D Program of China (No. 2018YFB1801105); the Key Areas of Research and Development Program of Guangdong Province, China (No. 2018B010114001); the Fundamental Research Funds for the Central Universities (No. ZYGX2019Z022); and the Programme of Introducing Talents of Discipline to Universities (No. B20064)