TLDR: VMask is a novel framework designed to protect sensitive label data in Vertical Federated Learning (VFL) from powerful Model Completion (MC) attacks. It employs a layer masking technique using Secret Sharing to disrupt the correlation between input data and intermediate outputs, effectively reducing label inference accuracy to a random guessing level. VMask also features a strategic layer selection process to minimize computational overhead and offers a tunable privacy budget, allowing defenders to flexibly control privacy levels without significantly sacrificing model accuracy or incurring impractical costs, a common challenge for existing defense methods.
Vertical Federated Learning (VFL) has emerged as a powerful approach for collaborative AI model training, allowing multiple parties to build a shared model without directly sharing their private datasets. This decentralized paradigm is generally considered privacy-preserving, as only intermediate information, like feature embeddings and gradients, is exchanged. However, recent research has unveiled a significant vulnerability: label inference attacks, particularly the Model Completion (MC) attack, which poses a severe threat to the sensitive label information held by one of the participants, known as the active party.
Existing defense mechanisms against the MC attack often face a dilemma: they either compromise the model’s accuracy or demand impractical computational resources. Some methods, while providing strong privacy, are too slow for real-world deployment, while others that are efficient fail to adequately protect labels. Furthermore, most current solutions lack the flexibility for defenders to adjust the level of label privacy based on their specific needs and available resources.
Introducing VMask: A Novel Approach to Label Privacy
To address these critical challenges, researchers have proposed VMask, a new label privacy protection framework. VMask tackles the MC attack by introducing a unique concept called ‘layer masking’. The core idea behind VMask is to disrupt the strong correlation between input data and the intermediate outputs within the attacker’s model. It achieves this by applying a cryptographic technique called Secret Sharing (SS) to mask specific layer parameters in the attacker’s model.
VMask’s design incorporates several key innovations:
-
Secret Sharing for Accuracy Preservation: By using Secret Sharing, VMask randomizes the parameters of selected layers. Crucially, SS maintains the integrity of linear computations, ensuring that the overall model performance for the main task is not significantly degraded, even while the extracted feature embeddings are randomized to thwart attacks.
-
Strategic Layer Selection: Not all layers contribute equally to label leakage. VMask includes a clever strategy to identify and mask only the ‘critical layers’ – those that have the most significant impact on label inference accuracy. This selective masking dramatically reduces the computational and communication overhead that would arise from masking the entire model, making the approach much more practical.
-
Tunable Privacy Budget: VMask is the first framework to offer a ‘tunable privacy budget’ to defenders. This allows the active party to flexibly control the desired level of label privacy. It works by employing a ‘shadow model’ that mirrors the attacker’s model. This shadow model is trained locally by the active party using a small auxiliary dataset. By simulating the MC attack on this shadow model, the active party can estimate label leakage and dynamically adjust which layers to mask until the leakage falls within the predefined privacy budget.
How VMask Operates
The VMask framework integrates seamlessly into a typical VFL system. In each training epoch, a layer masking module masks critical layers identified by a layer selection module. A secure model update procedure then trains these masked layers using Secret Sharing, while other layers are trained in plaintext. Concurrently, a shadow model update procedure trains a local shadow model. Based on the shadow model’s performance and estimated label leakage, the layer selection module determines which layers need masking for the next epoch. This iterative process continues until the VFL model converges.
Also Read:
- Navigating the Path to Trustworthy Federated Learning: A Comprehensive Overview
- VLA-Mark: Securing AI-Generated Multimodal Content with Vision-Aligned Watermarks
Performance and Impact
Extensive evaluations of VMask were conducted across five different model architectures and 13 diverse datasets, comparing it against 12 other defense methods. The results are compelling: VMask consistently achieves the best privacy-utility trade-off. It successfully thwarts the MC attack, reducing label inference accuracy to a level comparable to random guessing, while preserving the VFL model’s performance with only a marginal drop in accuracy (e.g., an average drop of just 0.09% in Transformer-based models).
In terms of efficiency, VMask significantly outperforms cryptography-based methods, running up to 60,846 times faster in some cases. While it has a slightly higher runtime than standard VFL, the increase is minimal (e.g., only 1.8 times for large Transformer models), making it generally acceptable for practical deployment. VMask also demonstrates robustness to the size and distribution of the auxiliary dataset, proving effective even with small or out-of-distribution auxiliary data.
VMask represents a significant step forward in securing Vertical Federated Learning, offering a practical, effective, and tunable solution to protect sensitive label information from sophisticated inference attacks. For more in-depth technical details, you can refer to the full research paper available here.
