TLDR: LSHFed is a new federated learning framework that enhances robustness against attacks and improves communication efficiency. It uses Locally-Sensitive Hashing Gradient Mapping (LSHGM) to project high-dimensional gradients into compact, irreversible binary forms for accurate malicious gradient detection, drastically reducing communication overhead. Combined with a mask-based privacy scheme and a reputation-driven role election algorithm (ScoreQ-Hash), LSHFed maintains high model accuracy even with 50% malicious participants and achieves significant communication reductions, outperforming existing baseline methods.
Federated learning (FL) is a powerful approach that allows multiple participants to collaboratively train a machine learning model without sharing their raw, sensitive data. This decentralized method has gained significant traction, especially with the rise of large language models (LLMs) and fine-tuning techniques, as it enables privacy-preserving model enhancement and reduces resource consumption.
However, the distributed nature of FL also introduces several critical challenges. These include persistent privacy and security vulnerabilities, such as inference attacks (where attackers try to reconstruct private training data from gradient updates) and poisoning attacks (where malicious nodes upload altered gradients to degrade model performance or induce specific behaviors). Additionally, FL systems often face issues with data heterogeneity among clients and substantial communication overhead.
Existing solutions to these problems often come with trade-offs: some robust aggregation methods incur high computational costs, while privacy-preserving techniques like homomorphic encryption or differential privacy can be computationally intensive or degrade model utility. Communication-efficient strategies, such as gradient quantization, might compromise the accuracy of malicious update identification.
Introducing LSHFed: A Robust and Communication-Efficient Solution
To address these intertwined challenges, researchers have proposed LSHFed, a novel federated learning framework designed for both robustness against attacks and communication efficiency. LSHFed aims to enhance aggregation reliability and privacy preservation simultaneously, even in environments where trust is limited.
At the heart of LSHFed is a mechanism called LSHGM, or Locally-Sensitive Hashing Gradient Mapping. This innovative technique projects high-dimensional gradient updates into compact, irreversible binary representations. By doing so, LSHGM can accurately detect and filter out malicious gradients using only these hashed forms, significantly reducing the risk of privacy leakage and drastically cutting down on data transmission overhead for verification. Experiments show LSHFed can achieve up to a 1000x reduction in gradient verification communication compared to methods that use full gradients.
How LSHFed Works
LSHFed operates through three main components:
1. LSHGM Malicious Gradient Detection: This algorithm is designed to be highly sensitive to malicious activity while being incredibly efficient. It generates multiple random hyperplane vectors and segments gradient matrices into column vectors. Each column vector is then projected onto these hyperplanes, resulting in a binary bit string. This bit string mapping is irreversible, meaning the original gradient cannot be recovered from it, thus protecting privacy. By comparing these bit strings with a benchmark (from the previous round or a trusted entity), LSHGM can identify and filter out malicious gradients based on their Hamming distance (the number of differing bits).
2. Mask-Based Privacy Protection Scheme: Before each training round, local trainers (LTs) within the same group collaboratively generate privacy masks. These masks are added to their local gradient updates. The aggregation nodes (AGs) then subtract a predefined total mask sum during aggregation, effectively removing the noise and ensuring that individual gradients remain confidential. This scheme guarantees information-theoretic privacy under an honest-majority assumption.
3. ScoreQ-Hash Role Election: This mechanism dynamically assigns roles (Local Trainer or Aggregator) to participants based on a reputation score. Nodes with higher scores, reflecting past reliability and good behavior (e.g., low LSHGM distance), have a greater probability of being selected. This system injects fresh randomness each round, making it difficult for malicious nodes to consistently influence the process and ensuring that honest nodes eventually participate.
Also Read:
- Collaborative AI Strengthens Ransomware Defense While Protecting Data Privacy
- Distributing Intelligence: How Networked AI Experts Power Mobile Devices
Security and Performance
LSHFed provides strong security guarantees:
- Gradient Privacy: Individual gradients are protected, with only the final, noise-free global model being revealed.
- Robustness: Malicious gradient injections and model-poisoning attempts are detected and excluded in every training round, ensuring the model converges to a reliable state.
- Liveness: Honest nodes are guaranteed eventual participation, preventing adversarial denial-of-service.
Extensive experiments on datasets like MNIST, CIFAR-10, and F-MNIST demonstrate LSHFed’s superior performance. It consistently achieves high model accuracy across various attack scenarios, maintaining performance comparable to benign (no attack) conditions even when up to 50% of participants are collusive adversaries. This resilience significantly surpasses other baseline methods, which often see a collapse in accuracy under high adversary fractions. Furthermore, LSHFed proves robust in non-IID (non-independently and identically distributed) data settings and scales effectively with an increasing number of participating clients.
In conclusion, LSHFed offers a promising advancement in federated learning, providing a framework that is both highly robust against diverse attacks and remarkably efficient in communication. Its innovative use of locally-sensitive hashing for gradient verification, combined with robust privacy protection and a secure role election mechanism, paves the way for more secure and scalable collaborative AI training. You can read the full research paper for more technical details and experimental results: LSHFed: Robust and Communication-Efficient Federated Learning with Locally-Sensitive Hashing Gradient Mapping.
