Byzantine-Robust Decentralized Learning via ClippedGossip

Authors: Lie He, Sai Praneeth Karimireddy, Martin Jaggi
Published: 2022-02-03 | Updated: 2023-04-20

Source: https://arxiv.org/abs/2202.01545

PDF: https://arxiv.org/pdf/2202.01545

Labels Predicted by AI

収束分析 アルゴリズム設計 グループベースの堅牢性

Please note that these labels were automatically added by AI. Therefore, they may not be entirely accurate.
For more details, please see the About the Literature Database page.

Abstract

In this paper, we study the challenging task of Byzantine-robust decentralized training on arbitrary communication graphs. Unlike federated learning where workers communicate through a server, workers in the decentralized environment can only talk to their neighbors, making it harder to reach consensus and benefit from collaborative training. To address these issues, we propose a ClippedGossip algorithm for Byzantine-robust consensus and optimization, which is the first to provably converge to a O(δ_maxζ²/γ²) neighborhood of the stationary point for non-convex objectives under standard assumptions. Finally, we demonstrate the encouraging empirical performance of ClippedGossip under a large number of attacks.