TLDR: Grab is a new gradient inversion attack that can recover a large portion of private training data (up to 92.9%) from language models trained using federated learning. It overcomes previous limitations by using a hybrid optimization approach, including dropout mask learning and beam search, making it effective in practical settings and resilient to current defenses.
The world of artificial intelligence is rapidly evolving, with language models becoming increasingly sophisticated and integral to many applications. A key method for training these powerful models, especially when dealing with sensitive data, is Federated Learning (FL). FL allows multiple parties to collaboratively train a model without directly sharing their raw data, instead exchanging only model updates or gradients. While this approach enhances data privacy, new research reveals a significant vulnerability: gradient inversion attacks.
Gradient inversion attacks are a type of privacy breach where an adversary attempts to reconstruct a client’s private training data by analyzing the gradients shared during the FL process. This threat has been well-documented in continuous domains like vision models, where attackers can effectively reconstruct images. However, applying these attacks to language models has historically been challenging due to the discrete nature of text data (tokens) and the reliance on impractical training settings.
A new study, titled “Uncovering Gradient Inversion Risks in Practical Language Model Training,” introduces a novel, domain-specific gradient inversion attack called **Grab** (gradient inversion with hybrid optimization). This research, conducted by Xinguo Feng, Zhongkui Ma, Zihan Wang, Eu Joe Chegne, Mengyao Ma, Alsharif Abuadbba, and Guangdong Bai, addresses the limitations of previous attacks, making gradient inversion a much more practical and potent threat to language models in FL scenarios.
Grab’s innovation lies in its two alternating optimization processes: a continuous optimization and a discrete optimization. The continuous optimization focuses on improving token recovery by simultaneously optimizing dropout masks between layers. Dropout is a common regularization technique that introduces randomness, making gradient inversion difficult. Grab’s ‘dropout mask learning’ technique effectively counteracts this noise, aiding in more accurate gradient matching.
The discrete optimization, on the other hand, tackles the challenge of correctly sequencing the recovered tokens. Previous methods struggled with restoring token order due to the dominance of token embedding over positional embedding. Grab employs a systematic approach called beam search for reordering. This method explores the search space by iterating through each position in a sequence and utilizing the interconnections between adjacent tokens, allowing for the recovery of significantly longer and more coherent sequences. Crucially, Grab’s beam search also incorporates padding tokens, dynamically adjusting sequence lengths without needing prior knowledge of exact lengths in the batch – a common, often impractical, assumption in older attacks.
The researchers conducted extensive evaluations, demonstrating Grab’s remarkable efficacy. In benchmark settings, Grab achieved up to a 92.9% recovery rate of private training data, outperforming existing attack strategies by notable margins (up to 28.9% improvement). More importantly, Grab maintained its high effectiveness in practical settings, which included frozen embedding layers and activated dropout – conditions that typically hinder other attacks. Here, Grab showed an even more significant improvement, up to 48.5% over the best baseline.
The study also explored Grab’s resilience against proactive defenses like gradient noise and gradient pruning. While these defenses aim to perturb gradients and mitigate attacks, Grab demonstrated strong resistance, often recovering substantial portions of data even when defenses significantly compromised the model’s utility. This highlights that current defense mechanisms may not be sufficient against sophisticated attacks like Grab.
Also Read:
- Adaptive Prompt-tuning for Efficient Federated Learning
- ZS-PAG: Efficient Data Removal from AI Models in Zero-Shot Scenarios
This groundbreaking work underscores the urgent need for the AI community to develop more robust defense mechanisms for federated learning, especially for language models. Grab not only sets a new baseline for gradient inversion attacks but also provides a valuable step forward in understanding the privacy threats in the emerging FL training mode of language models. For more technical details, you can refer to the full research paper available at https://arxiv.org/pdf/2507.21198.
