These labels were automatically added by AI and may be inaccurate. For details, see About Literature Database.
Abstract
Robust distributed learning algorithms aim to maintain reliable performance
despite the presence of misbehaving workers. Such misbehaviors are commonly
modeled as Byzantine failures, allowing arbitrarily corrupted communication, or
as data poisoning, a weaker form of corruption restricted to local training
data. While prior work shows similar optimization guarantees for both models,
an important question remains: How do these threat models impact
generalization? Empirical evidence suggests a gap, yet it remains unclear
whether it is unavoidable or merely an artifact of suboptimal attacks. We show,
for the first time, a fundamental gap in generalization guarantees between the
two threat models: Byzantine failures yield strictly worse rates than those
achievable under data poisoning. Our findings leverage a tight algorithmic
stability analysis of robust distributed learning. Specifically, we prove that:
(i) under data poisoning, the uniform algorithmic stability of an algorithm
with optimal optimization guarantees degrades by an additive factor of
$\varTheta ( \frac{f}{n-f} )$, with $f$ out of $n$ workers misbehaving; whereas
$\textit{(ii)}$ under Byzantine failures, the degradation is in $\Omega \big(
\sqrt{ \frac{f}{n-2f}} \big)$.