TLDR: MASK (Modular Adaptive Sanitization Kit) is a new framework that uses large language models (LLMs) to detect phone scams while protecting user privacy. It allows users to customize how much sensitive information is removed from conversations before being analyzed by LLMs. The framework offers various sanitization strategies, with PII Masking Anonymization proving most effective at balancing privacy protection with accurate scam detection, ensuring sensitive data is removed without compromising the ability to identify fraudulent calls.
Phone scams continue to be a significant global threat, causing immense financial and psychological damage. In recent years, large language models (LLMs) have shown great promise in detecting these scams by analyzing transcribed phone conversations. However, this powerful capability comes with a major privacy concern: these conversations often contain highly sensitive personal information that could be exposed when sent to third-party LLM service providers for processing.
Addressing this critical challenge, researchers from The Hong Kong Polytechnic University and The Education University of Hong Kong have introduced MASK (Modular Adaptive Sanitization Kit). MASK is an innovative framework designed to harness the power of LLMs for phone scam detection while rigorously preserving user privacy. It offers a flexible and user-centric approach, allowing individuals to dynamically adjust privacy settings based on their personal preferences and the context of the conversation.
Understanding MASK: A Flexible Privacy Framework
MASK stands out by moving beyond the traditional “one-size-fits-all” privacy rules. It recognizes that privacy needs are subjective and can change over time. The framework is built with a pluggable architecture, meaning it can easily incorporate various sanitization methods, from basic keyword-based techniques for users who prioritize maximum privacy to more advanced neural approaches for those who need higher detection accuracy.
The core of MASK comprises three main components:
- Privacy Preference Adapter: This acts as the control center, translating a user’s desired privacy level into specific sanitization strategies. Users can set a “risk tolerance” parameter, similar to how one might adjust the “temperature” in an LLM. A low risk tolerance means MASK will apply very aggressive sanitization, sending highly protected, abstract data to the cloud. A higher risk tolerance allows for more selective masking, preserving more conversational context while still protecting explicit personal identifiers.
- Modular Sanitization Layer: This layer houses a diverse collection of privacy-preserving methods, organized as interchangeable modules. MASK was initially implemented with four key strategies to offer a range of options matching different user risk tolerances.
- Extensible Plugin Architecture: This forward-looking design allows new sanitization techniques to be seamlessly integrated into MASK. It supports future advancements, from simple rule-based methods to complex neural network approaches, ensuring the framework can evolve with new privacy-preserving technologies.
Sanitization Strategies in Detail
The research paper details four primary sanitization strategies implemented within MASK:
- TF-IDF Keyword Representation: This method converts conversation transcripts into numerical vectors based on the frequency of selected keywords. It focuses on words indicative of scam content, preserving critical semantic information while minimizing privacy risk by not sending the full text.
- PII Statistical Representation: After identifying Personally Identifiable Information (PII) like names, phone numbers, and addresses using both pattern matching and neural entity recognition, this strategy transforms each utterance into a vector representing the count of different PII categories.
- PII Masking Anonymization: Similar to the statistical representation in identifying PII, this method replaces all detected PII entities with standardized placeholders (e.g., [NAME], [PHONE]). The anonymized transcript retains the conversational structure and context, making it highly effective for balancing privacy and detection accuracy.
- Transcript Summarization: This approach condenses entire transcripts into concise, high-level summaries, omitting extraneous details and all personally sensitive content. It leverages local, lightweight LLMs (like Qwen3-4b) on the user’s device to generate privacy-preserving summaries before any cloud-based analysis.
Balancing Privacy and Detection Accuracy
The effectiveness of these strategies was evaluated based on two key metrics: PII Removal Rate (PRR) for privacy preservation and Semantic Retention Rate (SRR) for maintaining the necessary information for scam detection. Experiments were conducted using several advanced LLMs, including ChatGPT-4o, Gemini-2.5-Flash, Qwen2.5-MAX, and DeepSeek-V3.
The findings revealed a fundamental trade-off between privacy protection and semantic utility. Among the strategies, PII Masking Anonymization achieved the strongest balance. It successfully removed all identified sensitive information (PRR = 1.000) while largely retaining the semantic content (SRR = 0.861), resulting in detection performance almost identical to using the original, unsanitized text. This indicates that replacing explicit personal details with category placeholders is highly effective for both privacy and accurate scam detection.
In contrast, TF-IDF Keyword Representation and PII Statistical Representation, while achieving perfect PII removal, significantly degraded semantic content, leading to a noticeable drop in detection accuracy. Transcript Summarization offered a moderate level of privacy and semantic retention, suitable for stricter privacy or resource constraints, but with some compromise in detection performance.
Also Read:
- Evaluating PHI De-identification Models with Multi-Agent AI
- SentinelNet: A Decentralized Shield for Collaborative AI Systems
Future Directions
The research emphasizes that MASK can be further enhanced with trainable sanitization modules. By jointly optimizing privacy risk, semantic fidelity, and detection performance, the system could offer even more nuanced control over the privacy-utility balance. The paper also discusses the potential for on-device detection using lightweight LLMs, acknowledging current challenges in accuracy and contextual understanding for complex, real-time conversations, but highlighting it as a promising area for future development.
This work provides valuable insights and practical guidance for developing privacy-conscious language understanding systems, particularly for sensitive real-world applications like phone scam detection. For more in-depth technical details, you can read the full research paper here.
