TLDR: This research addresses the underexplored issue of fairness in privacy protection within machine learning. It introduces a novel, efficient auditing mechanism (PA-ALOOA) to accurately measure approximate worst-case group privacy risks, revealing significant disparities overlooked by previous methods. Building on the finding that groups contributing more to model updates face higher privacy risks, the paper proposes an enhanced DP-SGD algorithm (DP-SGD-S) with adaptive group-specific gradient clipping. Experiments demonstrate that DP-SGD-S effectively mitigates these privacy risk disparities, promoting more equitable privacy protection in AI systems, often with a manageable trade-off in model accuracy.
Artificial intelligence (AI) and machine learning (ML) are increasingly integrated into various aspects of our lives, from healthcare to finance. While these technologies offer immense benefits, they also bring forth critical ethical concerns, particularly regarding data privacy and algorithmic fairness. A significant, yet often overlooked, challenge lies at the intersection of these two issues: ensuring that AI systems provide fair and equitable privacy protection across different groups of people.
Existing research has made strides in both privacy-aware ML and fairness-aware ML independently. However, the fairness of privacy protection itself—meaning whether all groups experience similar levels of privacy risk—has remained largely underexplored. Previous methods for assessing group privacy risks typically rely on average-case scenarios, which might underestimate the true risks faced by specific groups and, consequently, the disparities in privacy protection. Furthermore, more rigorous methods for evaluating worst-case privacy risks are often too computationally intensive for practical use.
A New Way to Measure Privacy Risk
To address these limitations, a recent research paper introduces a novel approach to more accurately measure privacy risks. The researchers propose an innovative “membership inference game” that allows for the efficient auditing of approximate worst-case privacy risks for individual data records. This method, referred to as Approximate Leave-One-Out Attack (PA-ALOOA), significantly improves computational efficiency by auditing multiple data samples simultaneously, unlike its computationally expensive predecessor, the Leave-One-Out Attack (LOOA), which audits one sample at a time.
The experimental results demonstrate that PA-ALOOA provides a more stringent and reliable measurement of group privacy risks. It effectively uncovers more pronounced privacy inequalities between groups compared to older, average-case auditing methods. The paper defines key metrics to quantify these risks: Individual Privacy Risk (IPR) for single data points, Group Privacy Risk (GPR) for subsets of data belonging to a specific group, and Group Privacy Risk Parity (GPRP), which measures the disparity between the most and least vulnerable groups.
Through extensive experiments, the researchers found that their auditing method consistently revealed higher GPR values and more accurately assessed GPRP across various machine learning models (Logistic Regression, Multilayer Perceptron, and Convolutional Neural Networks) and datasets. This indicates that existing ML algorithms often exhibit significant unfairness in privacy risks across groups, and even differentially private ML (DPML) algorithms, while binding the magnitude of disparities, still leave a noticeable gap.
Also Read:
- The Agentic Lakehouse: Enabling Safe AI-Driven Data Pipeline Management
- Unpacking Bias: How AI Medical Guidance Varies by Patient Demographics
Mitigating Privacy Disparities with Adaptive Clipping
Having established a more robust way to measure privacy unfairness, the researchers then focused on mitigating this issue. They observed a strong correlation: groups that contribute more significantly to model updates during training (indicated by larger gradient norms) are more susceptible to higher privacy leakage risks. This insight, inspired by techniques used in differential privacy auditing, led to an enhancement of the standard Differentially Private Stochastic Gradient Descent (DP-SGD) algorithm.
The proposed algorithm, called DP-SGD-Scale (DP-SGD-S), incorporates an adaptive group-specific gradient clipping strategy. Instead of applying a uniform clipping bound to all groups, DP-SGD-S adaptively sets different clipping bounds for each group based on its relative contribution to the overall gradient. Groups with higher contributions receive stricter clipping bounds, thereby limiting their influence on model updates and reducing their privacy leakage risks. Conversely, groups with smaller contributions are assigned larger clipping bounds. This adaptive approach helps to balance privacy protection across different groups.
Experimental validation confirmed that DP-SGD-S effectively reduces the disparity in group privacy risks across diverse datasets, including MNIST, Adult, Law, and UTKFace. While this enhancement might introduce a slight trade-off in model accuracy in some cases (e.g., a modest 2% drop for image datasets), this is often an acceptable compromise for achieving enhanced fairness in privacy protection. The study also showed that DP-SGD-S generally improves privacy fairness without increasing the privacy risk for already well-protected groups, thus avoiding a “leveling down” effect.
This research marks a significant step towards building more trustworthy and ethical AI systems by ensuring that privacy protection is not only strong but also fair across all demographic groups. For more in-depth information, you can read the full paper here.
