Teach LLMs to Phish: Stealing Private Information from Language Models

Authors: Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, Prateek Mittal
Published: 2024-03-01

Source: https://arxiv.org/abs/2403.00871

PDF: https://arxiv.org/pdf/2403.00871

AIにより推定されたラベル

バックドア攻撃 フィッシング検出 プロンプトインジェクション

※ こちらのラベルはAIによって自動的に追加されました。そのため、正確でないことがあります。
詳細は文献データベースについてをご覧ください。

Abstract

When large language models are trained on private data, it can be a significant privacy risk for them to memorize and regurgitate sensitive information. In this work, we propose a new practical data extraction attack that we call “neural phishing”. This attack enables an adversary to target and extract sensitive or personally identifiable information (PII), e.g., credit card numbers, from a model trained on user data with upwards of 10 success rates, at times, as high as 50 adversary can insert as few as 10s of benign-appearing sentences into the training dataset using only vague priors on the structure of the user data.