TLDR: A new research paper introduces “privacy profiles,” natural language instructions that allow users to control what personal data Large Language Models (LLMs) can access. Their framework uses a local LLM to rewrite queries, hiding sensitive details before sending them to a more powerful external LLM. They also release PEEP, a multilingual dataset for this research, annotated with various types of private information. Experiments show this approach improves response quality while significantly reducing data leakage, though challenges remain in protecting implicitly shared information and balancing strict privacy with task success.
In an era where Large Language Models (LLMs) are becoming ubiquitous, accessed primarily through commercial APIs, a critical question arises: how can users maintain control over their personal data? A recent research paper, “Controlling What You Share: Assessing Language Model Adherence to Privacy Preferences”, delves into this challenge, proposing an innovative framework that empowers users to define their privacy preferences using simple natural language instructions.
Understanding the Challenge of LLM Privacy
The widespread use of LLMs often means users expose their data to service providers, leading to concerns about data governance and individual autonomy. While Personally Identifiable Information (PII) like names and addresses are obvious privacy concerns, the scope of sensitive data is far broader. It can include financial records, proprietary code, or even personal hobbies and habits, depending on the context. What one user considers private, another might willingly share. This nuanced understanding of privacy, often referred to as Contextual Integrity, highlights the need for flexible, user-defined controls rather than rigid, predefined categories.
Introducing Privacy Profiles: A User-Centric Approach
The core of this research is the concept of ‘privacy profiles’ – natural language specifications that articulate a user’s preferences on what information should or should not be revealed. The proposed framework operates as a two-tiered system:
- A **local, trusted LLM (ML)**, which can be a smaller, on-device model, receives the user’s original query along with their privacy profile.
- A **rejector module** within the local LLM determines if the query can be safely paraphrased while respecting privacy. If not, the local model handles the query directly.
- If safe, a **paraphraser module** generates a ‘Privacy Compliant Query’ (PCQ), hiding sensitive details specified by the user.
- This PCQ is then sent to a more powerful, **untrusted external LLM (ME)**, typically a commercial API.
- Finally, an **aggregator module** integrates the external LLM’s response with the original query to produce a final answer, ensuring protected information is not reintroduced unless authorized.
This pipeline aims to strike a balance between privacy and the quality of responses, allowing users to leverage powerful external models without compromising their data.
The PEEP Dataset: A Foundation for Privacy Research
To support this research, the authors introduce PEEP (Prompts, Extracted Entities with Privacy), a multilingual dataset comprising 15,282 real user queries from the Wildchat dataset. What makes PEEP unique is its comprehensive annotation of private content, extending beyond traditional PII to include ‘soft’ personal information like hobbies, habits, religion, and personal connections. Each query in PEEP is paired with a synthetically generated privacy profile, created using various stylistic tones to simulate diverse user preferences. The dataset underwent rigorous anonymization to protect original user identities.
Experimental Insights: Balancing Privacy and Performance
The researchers evaluated the ability of lightweight LLMs (ranging from 3B to 8B parameters) to interpret and adhere to these privacy profiles, using GPT-4o and GPT-4o-mini as the external models. Key findings include:
- All tested LLMs showed improved response quality when utilizing the privacy-conscious pipeline compared to local processing, while also offering significantly better protection than traditional PII removal tools.
- The system demonstrated an ability to differentiate between protected and authorized attributes, meaning it selectively disclosed information in alignment with privacy profiles.
- While the best-performing model (Llama 8b) achieved a notable reduction in leakage of protected attributes (9.5% LeakPRO), this still indicates room for improvement, especially for highly sensitive data.
- Certain attributes, such as languages, gender, or personal habits, proved harder to protect due to their implicit textual signals within queries.
Simulating User Privacy Preferences
To further understand how privacy preferences influence model behavior, the study simulated different user personas: a ‘private user’ (minimal sharing), an ‘e-commerce user’ (sharing easily protected data like credit card numbers), and a ‘medical user’ (sharing harder-to-protect attributes like health status). The results showed that stricter privacy constraints often led to higher rejection rates (where the local model decided not to forward the query), but paradoxically, also resulted in more effective enforcement of privacy. This highlights the trade-off: greater privacy can sometimes come at the cost of overall task success, but privacy profiles offer a way for users to manage this balance.
Also Read:
- Unveiling cRID: A New AI Framework for Enhanced Privacy in Person Re-Identification
- Boosting LLM Agent Safety with Causal Influence Diagrams
Looking Ahead: Addressing Current Limitations
The research acknowledges limitations, particularly the reliance on numerous calls to large LLMs for both processing and evaluation, suggesting future work with more lightweight models and supervised fine-tuning. The synthetic nature of the privacy profiles also presents a challenge regarding their diversity and realism, though efforts were made to mitigate this through manual crafting and varied stylistic tones.
Overall, this work lays crucial groundwork for future privacy research, emphasizing the importance of user autonomy in managing personal data when interacting with advanced AI systems.
