These labels were automatically added by AI and may be inaccurate. For details, see About Literature Database.
Abstract
Recently, large language models (LLMs) have garnered widespread attention for
their exceptional capabilities. Prompts are central to the functionality and
performance of LLMs, making them highly valuable assets. The increasing
reliance on high-quality prompts has driven significant growth in prompt
services. However, this growth also expands the potential for prompt leakage,
increasing the risk that attackers could replicate original functionalities,
create competing products, and severely infringe on developers' intellectual
property. Despite these risks, prompt leakage in real-world prompt services
remains underexplored.
In this paper, we present PRSA, a practical attack framework designed for
prompt stealing. PRSA infers the detailed intent of prompts through very
limited input-output analysis and can successfully generate stolen prompts that
replicate the original functionality. Extensive evaluations demonstrate PRSA's
effectiveness across two main types of real-world prompt services.
Specifically, compared to previous works, it improves the attack success rate
from 17.8% to 46.1% in prompt marketplaces and from 39% to 52% in LLM
application stores, respectively. Notably, in the attack on "Math", one of the
most popular educational applications in OpenAI's GPT Store with over 1 million
conversations, PRSA uncovered a hidden Easter egg that had not been revealed
previously. Besides, our analysis reveals that higher mutual information
between a prompt and its output correlates with an increased risk of leakage.
This insight guides the design and evaluation of two potential defenses against
the security threats posed by PRSA. We have reported these findings to the
prompt service vendors, including PromptBase and OpenAI, and actively
collaborate with them to implement defensive measures.
External Datasets
prompt dataset D (approximately 50 prompts from each of the 18 popular categories)
360 prompts currently for sale from PromptBase
input-output pairs collected from 100 popular GPTs in OpenAI's GPT Store