TLDR: This research paper provides a systematic analysis of the challenges in making Federated Learning (FL) truly trustworthy, aligning it with the seven requirements of Trustworthy Artificial Intelligence (TAI). It details the progress made, current research trends, and remaining gaps across areas like human oversight, technical robustness, privacy, transparency, fairness, societal well-being, and accountability. The paper emphasizes that while FL inherently addresses privacy, its distributed nature introduces complexities in meeting other TAI requirements, and suggests that integrating ‘Collective Intelligence’ could be key to future advancements.
In today’s rapidly evolving digital landscape, Artificial Intelligence (AI) systems are becoming increasingly integrated into sensitive areas like healthcare, finance, and law enforcement. This widespread adoption has highlighted a crucial need for AI to be not just intelligent, but also trustworthy. This is where the concept of Trustworthy Artificial Intelligence (TAI) comes into play, outlining a set of ethical, legal, and technical standards to ensure AI aligns with human values and societal expectations.
One promising AI approach that addresses privacy concerns is Federated Learning (FL). Unlike traditional AI where all data is sent to a central server for training, FL allows models to be trained on decentralized devices, such as smartphones or hospitals, without the raw data ever leaving its source. This significantly enhances data privacy, making FL a natural fit for TAI principles.
However, despite its inherent privacy benefits, fully aligning FL with all aspects of TAI presents a unique set of challenges, largely due to its distributed nature. A recent research paper, “Challenges of Trustworthy Federated Learning: What’s Done, Current Trends and Remaining Work”, systematically explores these obstacles, categorizing them according to the seven key requirements of TAI as defined by the European Commission.
Human Agency and Oversight
This requirement emphasizes that humans should remain in control of AI systems. In FL, ensuring human involvement, especially when dealing with many distributed devices, is complex. Current trends focus on allowing human input at the local device level to create more robust models. However, integrating human oversight at the central server level and dealing with diverse data and systems across clients remain open challenges. Another hurdle is making the federated inference process, particularly in Vertical FL where different parts of a model are trained on different data, understandable to humans, often referred to as a “double black box” problem.
Technical Robustness and Safety
AI systems must be reliable and secure, even under unexpected conditions. FL faces threats like “poisoning attacks,” where malicious clients inject bad data or corrupted model updates. While defenses exist, it’s an ongoing “cat-and-mouse” game between new attacks and new defenses. Another issue is “free riders,” clients who benefit from the global model without contributing their fair share, which can degrade performance. Detecting data that is “out-of-distribution” (OOD) – data significantly different from what the model was trained on – is also harder in FL due to decentralized and private data.
Privacy and Data Governance
While FL is designed for privacy, it’s not entirely risk-free. “Inference attacks” can still allow adversaries to deduce sensitive information from shared model updates. Techniques like Differential Privacy (adding noise) and Secure Multi-Party Computation (securely combining data) are used, but they often come with a trade-off in model performance or computational cost. Another challenge is “model unlearning,” the ability to remove the influence of specific data or clients from a trained model, which is complicated by the distributed nature of FL and the lack of direct data access.
Transparency
AI systems should be understandable and traceable. In FL, the decentralized nature makes it hard to explain how decisions are made or trace model behavior. “Explainable AI (XAI)” in FL aims to provide insights, with current research focusing on using explainability to improve the model itself. “Causal AI,” which seeks to understand cause-and-effect relationships, is a new area in FL, promising more robust and interpretable models, but faces challenges with diverse and private data. Developing “models interpretable by design,” like decision trees, is also being explored, but their performance can suffer with heterogeneous FL data. Finally, “data provenance,” tracking the origin and lifecycle of data, is difficult to maintain consistently across a distributed FL network, though blockchain and watermarking are being investigated.
Diversity, Non-Discrimination & Fairness
AI systems must treat all individuals and groups equitably. “Data heterogeneity,” where client data varies significantly, is a major challenge in FL. This can lead to models that perform poorly for minority groups. Personalized FL, which adapts a global model to each client, and client clustering are current trends. However, these methods have limitations, such as requiring sufficient local data or potentially leaking sensitive information through grouping. Robust aggregation techniques are also being developed to mitigate the impact of outlier data.
Societal and Environmental Well-being
AI should contribute positively to society and the environment. FL, while beneficial for privacy, can incur high “communication costs” due to frequent model updates, especially for devices with limited network bandwidth. Efforts are focused on reducing communication rounds, decreasing the number of participants, and using model compression techniques. “Limited bandwidth” is a related challenge, with current methods adapting to fluctuating network conditions through dynamic compression and client selection. Future work needs to consider more realistic dynamic network scenarios and privacy implications.
Also Read:
- Spiking Neural Networks: Bringing Brain-Inspired AI to the Edge
- Unlocking Universal Ethics: How AI Could Reveal Hidden Moral Structures
Accountability
Clear mechanisms for responsibility, auditability, and verifiability are essential. “Model auditability” – verifying that a model functions as intended and complies with regulations – is hard in FL due to fragmented data and updates. Blockchain and Zero-Knowledge Proofs are being explored to maintain historical records and verify contributions. “Legal responsibility” is a significant, largely unaddressed challenge, as current laws like the EU AI Act struggle to define shared accountability between the FL server and clients. Data provenance, as mentioned under transparency, also plays a crucial role in accountability.
The paper concludes by highlighting that while FL offers a direct solution to privacy, substantial research and interdisciplinary collaboration are needed to make it truly trustworthy at scale. It also introduces “Collective Intelligence (CI)” as a new frontier, suggesting that trustworthiness can be co-constructed through inclusive and deliberative processes, moving beyond purely technical safeguards. This involves expanding human agency to include diverse stakeholders in the design, evaluation, and governance of FL systems, fostering a more socially legitimate and transparent decentralized AI.
