TLDR: The research paper introduces LEGO, a lightweight and efficient framework for unlearning multiple sensitive user attributes in recommender systems. It addresses the challenges of simultaneously handling multiple unlearning requests and adapting to dynamic privacy needs. LEGO uses a two-step process: ‘Embedding Calibration’ to remove specific attribute information from user embeddings, and ‘Flexible Combination’ to merge these calibrated embeddings, ensuring all sensitive attributes are protected while maintaining recommendation performance. Experiments show LEGO outperforms existing methods in unlearning effectiveness and efficiency.
In the evolving landscape of digital privacy, recommender systems face a significant challenge: how to protect sensitive user information while still providing personalized recommendations. The concept of the ‘Right to be Forgotten’ mandates that users should be able to withdraw their data, leading to the emergence of ‘recommendation unlearning’. While much research has focused on removing specific training data (input unlearning), the more complex task of ‘attribute unlearning’ – forgetting sensitive user characteristics like age or gender – has remained comparatively underexplored.
Existing methods for attribute unlearning primarily focus on single attributes and struggle with the dynamic, multi-faceted privacy demands of the real world. Imagine a user wanting to remove information about their age and gender, and later, their occupation. Current systems often fail to handle these multiple requests simultaneously or adapt efficiently when privacy requirements change.
The Core Challenges
Researchers have identified two key challenges in this area:
1. Simultaneous Unlearning (CH1): Existing methods struggle to unlearn multiple sensitive attributes at the same time. Sequential approaches might accidentally re-introduce previously unlearned attributes, while other multi-attribute methods can face conflicting optimization goals, leading to suboptimal results.
2. Dynamic Adaptability (CH2): Privacy needs are not static; they can increase, decrease, or alter over time. Current systems are inefficient in adapting to these changes, often requiring a complete re-execution of the unlearning process even for minor adjustments.
Introducing LEGO: A Novel Framework
To address these critical issues, a new framework called LEGO (Lightweight and Efficient Multiple-Attribute Unlearning Framework) has been proposed. LEGO tackles multi-attribute unlearning through a clever two-step process:
1. Embedding Calibration: This initial step focuses on individual attributes. For each sensitive attribute (e.g., age, gender), LEGO modifies the user’s digital representation (user embedding) to remove information related to that specific attribute. This is achieved by minimizing the ‘mutual information’ between the user embedding and the attribute, essentially making it harder for an attacker to infer the attribute from the embedding. Crucially, a ‘parameter space constraint’ is applied to ensure that these modifications don’t significantly harm the system’s ability to make good recommendations.
2. Flexible Combination: Once individual embeddings are calibrated for each sensitive attribute, LEGO combines them into a single, unified user embedding. This combination is ‘flexible’ because it only optimizes the weights assigned to each calibrated embedding, ensuring efficiency. The resulting combined embedding is designed to protect all sensitive attributes that require unlearning.
How LEGO Overcomes the Challenges
LEGO’s design directly addresses the identified challenges:
- For Simultaneous Unlearning (CH1): The framework provides a theoretical guarantee for effectively unlearning all attributes simultaneously. By first calibrating individual attributes and then intelligently combining them, LEGO ensures that the final user embedding protects all specified sensitive information without conflicts.
- For Dynamic Adaptability (CH2): LEGO is highly efficient. If new attributes need to be unlearned, the Embedding Calibration step can be performed in parallel for these new attributes. If existing requirements change (e.g., an attribute no longer needs protection), the Flexible Combination step can quickly construct a new embedding by simply adjusting the weights, without needing to re-run the entire unlearning process.
Also Read:
- Enhancing LLM Privacy: A New Approach to Federated Unlearning
- Improving Data Forgetting in Graph Models Through Influence and Preference
Experimental Validation
Extensive experiments were conducted on three real-world datasets (MovieLens 100K, MovieLens 1M, and KuaiSAR) and across three popular recommendation models (NCF, LightGCN, and MultVAE). The results consistently demonstrated LEGO’s superiority:
- Enhanced Unlearning: LEGO significantly reduced the ability of attackers to infer sensitive attributes, outperforming existing methods like DP, D2DFR, and AdvX.
- Preserved Recommendation Quality: Despite effectively unlearning attributes, LEGO maintained recommendation performance comparable to the original models, with minimal impact on metrics like Hit Ratio and Normalized Discounted Cumulative Gain.
- Superior Efficiency: LEGO proved to be significantly faster than other multi-attribute unlearning methods, especially when adapting to dynamic privacy requirements, making it a practical solution for real-world systems.
This research marks a significant step forward in attribute unlearning for recommender systems, offering a robust and efficient solution for protecting user privacy in an increasingly data-sensitive world. For more technical details, the full research paper can be accessed here.
