Aggressive or Imperceptible, or Both: Network Pruning Assisted Hybrid Byzantines in Federated Learning

AISTATS

How to backdoor federated learning

E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, V. Shmatikov

Published: 2020

arxiv

被引用数 14

Computing Research Repository (CoRR)

A Little Is Enough: Circumventing Defenses For Distributed Learning

Moran Baruch, Gilad Baruch, Yoav Goldberg

Published: 2019.2.17

Distributed learning is central for large-scale training of deep-learning models. However, they are exposed to a security threat in which Byzantine participants can interrupt or control the learning process. Previous attack models and their corresponding defenses assume that the rogue participants are (a) omniscient (know the data of all other participants), and (b) introduce large change to the parameters. We show that small but well-crafted changes are sufficient, leading to a novel non-omniscient attack on distributed learning that go undetected by all existing defenses. We demonstrate our attack method works not only for preventing convergence but also for repurposing of the model behavior (backdooring). We show that 20% of corrupt workers are sufficient to degrade a CIFAR10 model accuracy by 50%, as well as to introduce backdoors into MNIST and CIFAR10 models without hurting their accuracy

敵対的攻撃手法敵対的学習敵対的攻撃

ICLR

signsgd with majority vote is communication efficient and fault tolerant

J. Bernstein, J. Zhao, K. Azizzadenesheli, A. Anandkumar

Published: 2019

arxiv

被引用数 8

International Conference on Machine Learning (ICML)

Analyzing Federated Learning through an Adversarial Lens

Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, Seraphin Calo

Published: 2018.11.30

Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence. We explore a number of strategies to carry out this attack, starting with simple boosting of the malicious agent's update to overcome the effects of other agents' updates. To increase attack stealth, we propose an alternating minimization strategy, which alternately optimizes for the training loss and the adversarial objective. We follow up by using parameter estimation for the benign agents' updates to improve on attack success. Finally, we use a suite of interpretability techniques to generate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable. Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.

ポイズニング重み更新手法連合学習

Proceedings of the 29th International Conference on Machine Learning (ICML)

Poisoning attacks against support vector machines

Battista Biggio, Blaine Nelson, Pavel Laskov

Published: 2012

Advances in neural information processing systems

Machine learning with adversaries: Byzantine tolerant gradient descent

Blanchard, P., El Mhamdi, E. M., Guerraoui, R., Stainer, J.

Published: 2017

NDSS

Fltrust: Byzantine-robust federated learning via trust bootstrapping

X. Cao, M. Fang, J. Liu, N. Z. Gong

Published: 2021

Proceedings of Advances in Neural Information Processing Systems

Distributed training with heterogeneous data: Bridging median- and mean-based algorithms

X. Chen, T. Chen, H. Sun, S. Wu, M. Hong

Published: 2020

International Conference on Learning Representations

Progressive skeletonization: Trimming more fat from a network at initialization

P. de Jorge, A. Sanyal, H. Behl, P. Torr, G. Rogez, P. K. Dokania

Published: 2021

PODC '20

Genuinely distributed byzantine machine learning

E.-M. El-Mhamdi, R. Guerraoui, A. Guirguis, L. N. Hoang, S. Rouault

Published: 2020

ICLR

Distributed momentum for byzantine-resilient stochastic gradient descent

E.-M. El-Mhamdi, R. Guerraoui, S. Rouault

Published: 2021

arxiv

被引用数 1

International Conference on Machine Learning (ICML)

The Hidden Vulnerability of Distributed Learning in Byzantium

El Mahdi El Mhamdi, Rachid Guerraoui, Sébastien Rouault

Published: 2018.2.22

While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD against adversarial (Byzantine) workers sending poisoned gradients during the training phase. Some of these approaches have been proven Byzantine-resilient: they ensure the convergence of SGD despite the presence of a minority of adversarial workers. We show in this paper that convergence is not enough. In high dimension $d \gg 1$, an adver\-sary can build on the loss function's non-convexity to make SGD converge to ineffective models. More precisely, we bring to light that existing Byzantine-resilient schemes leave a margin of poisoning of $\Omega\left(f(d)\right)$, where $f(d)$ increases at least like $\sqrt{d~}$. Based on this leeway, we build a simple attack, and experimentally show its strong to utmost effectivity on CIFAR-10 and MNIST. We introduce Bulyan, and prove it significantly reduces the attackers leeway to a narrow $O( \frac{1}{\sqrt{d~}})$ bound. We empirically show that Bulyan does not suffer the fragility of existing aggregation rules and, at a reasonable cost in terms of required batch size, achieves convergence as if only non-Byzantine gradients had been used to update the model.

機械学習手法ポイズニング敵対的攻撃

Proceedings of the 37th International Conference on Machine Learning

Rigging the lottery: Making all tickets winners

U. Evci, T. Gale, J. Menick, P. S. Castro, E. Elsen

Published: 2020

arxiv

被引用数 1

USENIX Security Symposium

Local Model Poisoning Attacks to Byzantine-Robust Federated Learning

Minghong Fang, Xiaoyu Cao, Jinyuan Jia, Neil Zhenqiang Gong

Published: 2019.11.27

In federated learning, multiple client devices jointly learn a machine learning model: each client device maintains a local model for its local training dataset, while a master device maintains a global model via aggregating the local models from the client devices. The machine learning community recently proposed several federated learning methods that were claimed to be robust against Byzantine failures (e.g., system failures, adversarial manipulations) of certain client devices. In this work, we perform the first systematic study on local model poisoning attacks to federated learning. We assume an attacker has compromised some client devices, and the attacker manipulates the local model parameters on the compromised client devices during the learning process such that the global model has a large testing error rate. We formulate our attacks as optimization problems and apply our attacks to four recent Byzantine-robust federated learning methods. Our empirical results on four real-world datasets show that our attacks can substantially increase the error rates of the models learnt by the federated learning methods that were claimed to be robust against Byzantine failures of some client devices. We generalize two defenses for data poisoning attacks to defend against our local model poisoning attacks. Our evaluation results show that one defense can effectively defend against our attacks in some cases, but the defenses are not effective enough in other cases, highlighting the need for new defenses against our local model poisoning attacks to federated learning.

ポイズニング攻撃タイプモデル性能評価