AIセキュリティポータル K Program
ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries
Share
Abstract
Differential Privacy (DP) is a mathematical framework that is increasingly deployed to mitigate privacy risks associated with machine learning and statistical analyses. Despite the growing adoption of DP, its technical privacy parameters do not lend themselves to an intelligible description of the real-world privacy risks associated with that deployment: the guarantee that most naturally follows from the DP definition is protection against membership inference by an adversary who knows all but one data record and has unlimited auxiliary knowledge. In many settings, this adversary is far too strong to inform how to set real-world privacy parameters. One approach for contextualizing privacy parameters is via defining and measuring the success of technical attacks, but doing so requires a systematic categorization of the relevant attack space. In this work, we offer a detailed taxonomy of attacks, showing the various dimensions of attacks and highlighting that many real-world settings have been understudied. Our taxonomy provides a roadmap for analyzing real-world deployments and developing theoretical bounds for more informative privacy attacks. We operationalize our taxonomy by using it to analyze a real-world case study, the Israeli Ministry of Health's recent release of a birth dataset using DP, showing how the taxonomy enables fine-grained threat modeling and provides insight towards making informed privacy parameter choices. Finally, we leverage the taxonomy towards defining a more realistic attack than previously considered in the literature, namely a distributional reconstruction attack: we generalize Balle et al.'s notion of reconstruction robustness to a less-informed adversary with distributional uncertainty, and extend the worst-case guarantees of DP to this average-case setting.
The US census bureau adopts differential privacy
J. M. Abowd
Published: 2018
The 2010 Census confidentiality protections failed, here’s how and why
J. M. Abowd, T. Adams, R. Ashmead, D. Darais, S. Dey, S. L. Garfinkel, N. Goldschlag, D. Kifer, P. Leclerc, E. Lew, S. Moore, R. A. Rodr’iguez, R. N. Tadros, L. Vilhuber
Published: 2023
Reconstructing Training Data with Informed Adversaries
Borja Balle, Giovanni Cherubin, Jamie Hayes
Published: 1.13.2022
Safely expanding research access to administrative tax data: creating a synthetic public use file and a validation server
L. E. Burman, A. Engler, S. Khitatrakun, J. R. Nunns, S. Armstrong, J. Iselin, G. MacDonald, P. Stallworth
Published: 2019
Privacy harms
D. K. Citron, D. J. Solove
Published: 2022
Towards formalizing the GDPR’s notion of singling out
A. Cohen, K. Nissim
Published: 2020
SoK: Differential privacies
D. Desfontaines, B. Pej´o
Published: 2020
Confidence-ranked reconstruction of census microdata from published statistics
T. Dick, C. Dwork, M. Kearns, T. Liu, A. Roth, G. Vietri, Z. S. Wu
Published: 2023
Statistical Confidentiality: Principles and Practice
G. T. Duncan, M. Elliot, J.-J. Salazar-Gonz´alez
Published: 2011
Calibrating noise to sensitivity in private data analysis
Cynthia Dwork, Frank McSherry, Kobbi Nissim, Adam Smith
Published: 2006
Exposed! a survey of attacks on private data
C. Dwork, A. D. Smith, T. Steinke, J. Ullman
Published: 2017
Bounding training data reconstruction in dp-sgd
Jamie Hayes, Saeed Mahloujifar, Borja Balle
Published: 2023
Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays
N. Homer, S. Szelinger, M. Redman, D. Duggan, W. Tembe, J. Muehling, J. V. Pearson, D. A. Stephan, S. F. Nelson, D. W. Craig
Published: 2008
Negotiating privacy/utility trade-offs under differential privacy
G. Miklau
Published: 2022
A unified analysis of label inference attacks
A. Munoz Medina, T. Dick, C. Gentile, R. I. Busa-Fekete, M. Swanberg
Published: 2023
Adversary instantiation: Lower bounds for differentially private machine learning
Milad Nasr, Shuang Songi, Abhradeep Thakurta, Nicolas Papernot, Nicholas Carlini
Published: 2021
A survey of privacy attacks in machine learning
M. Rigaki, S. Garc´ıa
Published: 2024
Privacy auditing with one (1) training run
Thomas Steinke, Milad Nasr, Matthew Jagielski
Published: 2023
Weaving technology and policy together to maintain confidentiality
L. Sweeney
Published: 1997
Statistical Disclosure Control in Practice
L. Willenborg, T. Waal
Published: 1996
Differentially private SQL with bounded user contribution
R. J. Wilson, C. Y. Zhang, W. Lam, D. Desfontaines, D. Simmons-Marengo, B. Gipson
Published: 2020
Privbayes: private data release via bayesian networks
J. Zhang, G. Cormode, C. M. Procopiuc, D. Srivastava, X. Xiao
Published: 2014
Share