TLDR: This research introduces a novel method for generating privacy-preserving synthetic text, crucial for AI development in sensitive fields like healthcare and law. Moving beyond traditional differential privacy, the approach uses ‘control codes’ and the ‘Hiding In Plain Sight’ theory to guide Large Language Models. Two variants, In-Context Learning (ICL) with privacy enhancement and Prefix Tuning with masking, are proposed. Experiments show that while ICL offers strong direct leakage prevention, Prefix Tuning with masking achieves a superior balance between privacy protection and the utility of the generated text, making it a practical solution for real-world applications.
The responsible development and deployment of artificial intelligence, especially in sensitive areas like healthcare, social services, and law, hinges on effective text anonymization. Sharing or training AI models with sensitive data carries significant risks of privacy breaches. While tools for redacting direct identifiers (like names and addresses) have improved, they often fail to capture all sensitive information, leaving behind ‘residual’ or ‘quasi-identifying’ details that could still lead to re-identification.
Recently, synthetic text generation using Large Language Models (LLMs) has emerged as a promising alternative. The idea is to create realistic but anonymized text that scrambles identifying information. However, traditional methods involving differentially private fine-tuning, while offering privacy guarantees, often degrade the quality of the generated synthetic text. Moreover, maintaining differential privacy in text is challenging due to the unclear unit of privacy, sometimes leading to actual leakage.
A New Approach to Privacy-Preserving Synthetic Text
Researchers Zihao Zhao and Anjalie Field from Johns Hopkins University have introduced a novel methodology for generating privacy-preserving synthetic text. Their approach moves away from solely relying on differential privacy and instead draws on established de-identification practices and the ‘Hiding In Plain Sight’ (HIPS) theory. HIPS suggests that replacing real identifiers with realistic, fabricated surrogates can effectively obscure any accidentally leaked real information, making it harder to detect or exploit.
The core of their method involves using ‘control codes’ to guide the text generation process. These control codes represent categories of private information, such as names, dates, or locations, and their associated values. For real documents, private entities are extracted and grouped into these control codes. During the generation of synthetic text, fictional control codes—containing randomly sampled, falsified values—are used to direct the LLM to produce fake sensitive information.
Two Variants for Controllable Generation
The paper proposes two main variants for this controlled generation:
1. In-Context Learning (ICL): In this setup, the LLM is provided with a few examples of real text paired with their control codes, followed by a fictional control code. The model then generates new text based on the fictional code, learning from the provided examples. To further enhance privacy, an optional ‘privacy enhancement’ mechanism explicitly blocks the generation of sensitive tokens from the input examples, ensuring privacy levels at least as strong as the underlying de-identification system. While effective in preventing direct leaks, this method can sometimes lead to the model copying more content from the input examples.
2. Prefix Tuning (Fine-tuning): This variant involves fine-tuning the LLM using pairs of control codes and their corresponding real texts. At inference time, fictional control codes are used to generate synthetic text. This approach is made more robust with a custom masking strategy and a unique loss function. This loss function encourages the fine-tuned model to diverge from the base model on private tokens (enhancing privacy) while staying close to it on non-private content (preserving utility). This method generally results in more synthesized and less copied output, offering better protection for information not explicitly marked as identifiable.
Also Read:
- CURE: A Framework for Self-Correcting Language Model Unlearning
- Challenging LLM Ownership: New Research Exposes Weaknesses in Fingerprinting Methods
Balancing Privacy and Utility
The researchers conducted experiments on legal (Text Anonymization Benchmark – TAB) and clinical (MIMIC-III) datasets. They evaluated privacy using metrics like Private Information Presence Percentage (PIPP) and Entity Leakage Percentage (ELP), which measure how often private information appears in the generated text. Utility was assessed using perplexity (how well the model predicts text) and MAUVE (a metric for distributional similarity between generated and human text).
Key findings include:
- All proposed methods significantly improved privacy protection compared to a baseline without control codes.
- ICL with privacy enhancement achieved the strongest privacy protection, often reaching near-zero leakage, but sometimes resulted in higher ROUGE scores, indicating more direct copying from input examples.
- The prefix tuning with masking method demonstrated a better balance between privacy and utility. It produced more diverse and less memorized outputs, making it highly practical for real-world applications, even when dealing with imperfect de-identification models or a larger set of sensitive entities (including quasi-identifiers).
- Ablation studies showed the method’s robustness across different model scales (e.g., Llama 3.1-8B Instruct) and its ability to handle both direct and quasi-identifiers, though performance can decline with a significant increase in private terms.
This research offers a practical and effective solution for generating high-quality synthetic text in sensitive domains, addressing the critical need to balance data utility with robust privacy protection. For more technical details, you can read the full research paper here.
