Byzantine-Robust Decentralized Learning via ClippedGossip

Authors: Lie He, Sai Praneeth Karimireddy, Martin Jaggi
Published: 2022-02-03 | Updated: 2023-04-20

Source: https://arxiv.org/abs/2202.01545

PDF: https://arxiv.org/pdf/2202.01545

AIにより推定されたラベル

収束分析 アルゴリズム設計 グループベースの堅牢性

※ こちらのラベルはAIによって自動的に追加されました。そのため、正確でないことがあります。
詳細は文献データベースについてをご覧ください。

Abstract

In this paper, we study the challenging task of Byzantine-robust decentralized training on arbitrary communication graphs. Unlike federated learning where workers communicate through a server, workers in the decentralized environment can only talk to their neighbors, making it harder to reach consensus and benefit from collaborative training. To address these issues, we propose a ClippedGossip algorithm for Byzantine-robust consensus and optimization, which is the first to provably converge to a O(δ_maxζ²/γ²) neighborhood of the stationary point for non-convex objectives under standard assumptions. Finally, we demonstrate the encouraging empirical performance of ClippedGossip under a large number of attacks.