These labels were automatically added by AI and may be inaccurate. For details, see About Literature Database.
Abstract
When large language model (LLM) agents are increasingly deployed to automate
tasks and interact with untrusted external data, prompt injection emerges as a
significant security threat. By injecting malicious instructions into the data
that LLMs access, an attacker can arbitrarily override the original user task
and redirect the agent toward unintended, potentially harmful actions. Existing
defenses either require access to model weights (fine-tuning), incur
substantial utility loss (detection-based), or demand non-trivial system
redesign (system-level). Motivated by this, we propose DataFilter, a test-time
model-agnostic defense that removes malicious instructions from the data before
it reaches the backend LLM. DataFilter is trained with supervised fine-tuning
on simulated injections and leverages both the user's instruction and the data
to selectively strip adversarial content while preserving benign information.
Across multiple benchmarks, DataFilter consistently reduces the prompt
injection attack success rates to near zero while maintaining the LLMs'
utility. DataFilter delivers strong security, high utility, and plug-and-play
deployment, making it a strong practical defense to secure black-box commercial
LLMs against prompt injection. Our DataFilter model is released at
https://huggingface.co/JoyYizhu/DataFilter for immediate use, with the code to
reproduce our results at https://github.com/yizhu-joy/DataFilter.