TLDR: A new research paper introduces PPML-Hybrid, a federated learning method that allows clients to choose between Homomorphic Encryption (HE) or Differential Privacy (DP) based on their computational resources and privacy needs. This hybrid approach aims to overcome the accuracy limitations of DP and the high computational cost of HE, achieving high model accuracy comparable to HE-only methods while significantly reducing computation time. It offers a flexible solution for privacy-preserving machine learning, particularly for sensitive omics data in heterogeneous environments.
Omics data, which includes information like genomics, transcriptomics, and proteomics, is incredibly valuable for medical research, helping scientists understand disease mechanisms. However, this data also contains highly sensitive personal information, making its privacy protection paramount, especially when multiple institutions collaborate.
Traditional Artificial Intelligence (AI) and deep learning methods, while powerful for analyzing multi-omics data, pose significant privacy risks. Federated Learning (FL) emerged as a promising solution, allowing machine learning models to be trained collaboratively across different institutions without sharing the raw, sensitive data. Yet, FL alone isn’t a complete privacy guarantee; malicious actors can still infer sensitive information from the exchanged model updates.
To bolster privacy, two main techniques have been widely explored: Differential Privacy (DP) and Homomorphic Encryption (HE).
The Challenges with Existing Privacy Methods
Differential Privacy (DP) works by adding carefully calibrated noise to data or model updates. This provides a strong, mathematically rigorous privacy guarantee, making it difficult to determine if any single individual’s data was part of the training set. However, DP faces an inherent trade-off: stronger privacy (more noise) often leads to a decrease in the model’s predictive accuracy. For instance, a prior method called PPML-Omics, which combines DP with Decentralized Randomization, improved privacy amplification but still couldn’t fully escape this accuracy-privacy dilemma.
Homomorphic Encryption (HE), on the other hand, allows computations to be performed directly on encrypted data. This means that model updates can be aggregated by a central server without ever being decrypted, thus preserving privacy without introducing noise that degrades accuracy. The downside of HE is its substantial computational overhead, making it resource-intensive for clients, especially those with limited computing power.
Introducing PPML-Hybrid: A Flexible Solution
To address the limitations of both DP and HE, researchers have proposed a novel hybrid approach called PPML-Hybrid. This method aims to improve predictive accuracy while considering the diverse computational abilities of different clients in a federated learning network. The core idea is simple yet effective: clients autonomously choose between using HE or DP based on their local computational resources and privacy requirements.
In the PPML-Hybrid framework, the network consists of a central server and multiple participating clients. At the start of each training round, clients decide their privacy mode. Clients with high computational resources and stringent privacy needs might opt for HE, contributing noise-free updates. Meanwhile, clients with limited resources or less strict privacy requirements can choose DP, reducing their computational burden. This creates a flexible, asymmetric mix of privacy mechanisms.
The process involves clients performing local training, then encrypting their updates with HE or adding noise with DP before sending them to the server. The server then aggregates these updates separately from the HE and DP groups, decrypts the HE-encrypted sum, and combines both aggregates to update the global model. This dynamic selection allows for an optimized balance between privacy, model utility, and training time.
Also Read:
- Hybrid AI Approaches for Video Violence Detection
- Boosting LLM Performance with Implicit Federated In-Context Learning
Performance and Efficiency
Evaluations of PPML-Hybrid using a public spatial transcriptomics dataset of human breast cancer demonstrated significant advantages:
- Improved Accuracy: PPML-Hybrid achieved model accuracy (measured by Mean Squared Error or MSE) comparable to HE-only methods, which represent the theoretical upper limit of accuracy without noise. Crucially, it consistently outperformed DP-only methods (like PPML-Omics) under equivalent or even stricter privacy budgets. This indicates that the noise-free updates from HE clients effectively mitigate the accuracy degradation caused by DP.
- Reduced Computation Time: While HE-only methods are highly accurate, they incur substantial computational costs. PPML-Hybrid significantly reduced the computation time per round compared to HE-only approaches. Although it was slower than DP-only (due to the overhead of HE for some clients), its hybrid nature allows for a better overall balance, especially when clients have heterogeneous computing resources. The computation time increases as the proportion of HE clients grows, highlighting the trade-off that clients can manage based on their capabilities.
The PPML-Hybrid method offers a flexible architecture that paves the way for broader adoption of privacy-preserving machine learning in environments where computational resources vary widely. While the current research focused on omics data, the approach is applicable to other common datasets as well. Future work aims to extend client selection to consider more realistic threat scenarios. You can read the full research paper here.
