Federated learning with incremental clustering for heterogeneous data

International audienceFederated learning enables different parties to collaboratively build a global model under the orchestration of a server while keeping the training data on the clients devices. However, performance is affected when clients have heterogeneous data. To cope with this problem, we assume that despite data heterogeneity, there are groups of clients who have similar data distributions that can be clustered. In order to do so, previous approaches require all clients to send their parameters to the server simultaneously. However, this is unfeasible in real-world cross-device contexts where clients are numerous and have limited availability. To prevent such a bottleneck, we propose FLIC (Federated Learning with Incremental Clustering), which exploits the updates sent by clients at each round as they participate in federated training to cluster clients based on the received updates. Hence no additional communications between the server and the clients are necessary, other than what classical federated learning requires. We empirically demonstrate for various non-IID cases that our approach successfully splits clients into groups following the same data distributions. We also identify the limitations of FLIC by studying its capability to partition clients at the early stages of the federated learning process efficiently. We further address attacks on models as a form of data heterogeneity and empirically show that FLIC is a robust defense to poisoning attacks even when the proportion of malicious clients is higher than 50%