TLDR: A new research paper introduces CompLeak, a framework to evaluate privacy leakage in compressed deep learning models. It demonstrates that common compression techniques like pruning, quantization, and weight clustering significantly increase the risk of membership inference attacks. The framework’s variants, especially CompLeakMR which leverages multiple compressed models, show that compression exacerbates privacy leakage, highlighting the need for privacy-aware compression strategies in AI development.
Deep learning models, especially large language models (LLMs), are becoming increasingly powerful but also increasingly large, demanding significant computational resources and memory. To address this, a technique called model compression is widely used. This process reduces the size of these models, making them faster and more efficient for deployment on devices like smartphones and IoT devices. However, a new research paper titled “CompLeak: Deep Learning Model Compression Exacerbates Privacy Leakage” by Na Li, Yansong Gao, Hongsheng Hu, Boyu Kuang, and Anmin Fu, reveals a critical, often overlooked, downside: model compression can significantly worsen privacy leakage.
The paper introduces CompLeak, a pioneering framework designed to evaluate the privacy risks associated with three common model compression techniques: pruning, quantization, and weight clustering. These operations, supported by commercial frameworks like Google’s TensorFlow-Lite and Facebook’s PyTorch Mobile, are crucial for optimizing model efficiency. Yet, their impact on privacy has remained largely unexplored.
CompLeak investigates this privacy risk through the lens of Membership Inference Attacks (MIA). MIA is a technique where an attacker tries to determine if a specific data point was part of the dataset used to train a machine learning model. If successful, this can expose sensitive personal information. For instance, an attacker could infer that a person’s medical records were used in a clinical trial if they can determine their data was part of a model’s training set.
The researchers developed three variants of the CompLeak framework, each designed for different levels of attacker access to model versions:
CompLeakNR: The Baseline Assessment
This variant uses existing MIA methods to attack a single compressed model. Surprisingly, the study found that highly compressed models were generally less vulnerable to these baseline attacks than the original, uncompressed models. This is likely because extreme compression limits the model’s capacity, reducing its tendency to ‘overfit’ or memorize specific training data details.
CompLeakSR: Leveraging a Single Reference
CompLeakSR introduces a novel approach by using the original, uncompressed model as a reference alongside a single compressed model. The core insight here is that compression affects how a model processes training data members versus non-members differently. By comparing the outputs (specifically, the posterior probability distributions) of both the original and compressed models, CompLeakSR can capture these subtle changes. This method significantly amplifies the privacy leakage detection, proving that compression operations indeed pose an additional privacy threat.
Also Read:
- How Model Compression Can Impact AI Security
- Unmasking LLM Inputs: The Inversion Threat to Internal States
CompLeakMR: Exploiting Multiple Compressed Models
This is where the privacy risk becomes most pronounced. Model providers often release multiple compressed versions of a model, offering users different trade-offs between performance and size. CompLeakMR innovatively exploits information from multiple compressed models, with or without access to the original model. The researchers discovered that different compressed versions leak privacy in slightly different ways. By aggregating these distinct leakages – combining information like the models’ losses and the meta-posteriors from CompLeakSR attacks – CompLeakMR achieves the highest MIA performance. This demonstrates that the more compressed models an attacker can access, the greater the overall privacy leakage.
The extensive experiments conducted across seven diverse model architectures (including ResNet, BERT, and GPT-2) and six benchmark datasets consistently showed that CompLeakMR achieved the best MIA performance. This confirms the central finding: model compression exacerbates privacy leakage.
The study also delved into which data samples become vulnerable after compression, finding that most are sensitive ‘members’ of the training dataset, and often, these are the more important data points. Even when privacy defense mechanisms like Differential Privacy (DP-SGD) are applied, CompLeakSR still demonstrated superior attack performance compared to baseline MIAs, although the additional leakage decreased with stronger defenses.
The findings of the CompLeak research underscore an urgent need for developers and model providers to consider privacy implications when implementing model compression. The paper suggests mitigation strategies such as incorporating differential privacy, training with synthetic data, and reducing model overfitting before deploying compressed models through query APIs. This work serves as a crucial step towards understanding and addressing the hidden privacy costs of efficient deep learning. You can read the full research paper here.
