TLDR: FusionFM is a new framework that systematically evaluates and combines different AI foundation models for ophthalmic diagnosis. It benchmarks four leading models (RETFound, VisionFM, RetiZero, DINORET) on both eye diseases (glaucoma, DR, AMD) and systemic diseases (diabetes, hypertension) using retinal images. The study found that DINORET and RetiZero generally perform best, and a “Gating-based” fusion strategy can slightly improve predictions for certain conditions like glaucoma, AMD, and hypertension, though predicting systemic diseases in new patient groups remains challenging.
Artificial intelligence (AI) is rapidly transforming medical image analysis, and ophthalmology is no exception. Recent years have seen the emergence of several specialized AI models, known as foundation models (FMs), designed to analyze eye images for various conditions. However, a crucial question has remained: which of these models performs best, are they equally effective across different tasks, and can combining them lead to even better results?
A groundbreaking new study introduces FusionFM, a comprehensive framework designed to answer these very questions. This research systematically evaluates both individual and combined ophthalmic FMs, providing much-needed clarity in this evolving field. The study, titled FusionFM: Fusing Eye-specific Foundational Models for Optimized Ophthalmic Diagnosis, was conducted by a team of researchers including Ke Zou, Jocelyn Hui Lin Goh, and Yih Chung Tham, among others, from institutions across Singapore, the United Kingdom, China, and Switzerland.
Understanding FusionFM’s Approach
FusionFM serves as a robust evaluation suite, assessing the performance of AI models in two key areas: detecting ophthalmic diseases and predicting systemic diseases based on retinal imaging. For eye diseases, it focuses on common conditions like glaucoma, diabetic retinopathy (DR), and age-related macular degeneration (AMD). For systemic diseases, it investigates the prediction of diabetes and hypertension, conditions that often show tell-tale signs in the retina.
The study benchmarked four state-of-the-art ophthalmic FMs: RETFound, VisionFM, RetiZero, and DINORET. Each of these models employs distinct architectures and pretraining strategies, making a direct comparison essential. FusionFM used standardized datasets from multiple countries and evaluated performance using widely accepted metrics like AUC (Area Under the Receiver Operating Characteristic Curve) and F1 score.
The Power of Fusion
Beyond evaluating individual models, FusionFM introduces two novel fusion approaches: Gating-based and Router-based methods. These strategies aim to integrate the strengths of multiple FMs, allowing them to work together to produce more accurate diagnoses. The idea is that by combining different models, the system can leverage their complementary knowledge and reduce prediction uncertainties.
Key Findings and Insights
The research yielded several significant insights:
-
Top Performers: DINORET and RetiZero consistently demonstrated superior performance in both ophthalmic and systemic disease tasks. Notably, RetiZero showed stronger generalization capabilities when tested on external datasets, suggesting its robustness across diverse patient populations.
-
Benefits of Fusion: The Gating-based fusion strategy provided modest but consistent improvements in predicting glaucoma, AMD, and hypertension. This indicates that combining models can indeed enhance diagnostic accuracy for certain conditions, leveraging the collective intelligence of different FMs.
-
Challenges Remain: Despite these advancements, predicting systemic diseases, particularly hypertension in external patient groups, continues to be a challenging area. This highlights a need for further research and development in this specific domain.
The study also noted that RETFound, RetiZero, and DINORET generally outperformed VisionFM in systemic disease prediction. This could be attributed to factors like richer and more diverse pre-training datasets (as seen with RetiZero) or powerful underlying architectures (like DINORET’s DINOv2 backbone).
Also Read:
- RETFound Foundation Model Adapted for Precise Optic Disc Segmentation
- Advancing Diabetic Retinopathy Screening Through Responsible AI Development
Looking Ahead
The FusionFM study provides an evidence-based evaluation of ophthalmic FMs, underscoring the benefits of model fusion for optimized ophthalmic diagnosis. The findings offer valuable guidance for enhancing the clinical applicability of these advanced AI tools. Future work will focus on further analyzing model architectures and efficiency to improve transparency and interpretability, ultimately paving the way for broader clinical deployment and real-world impact in eye care.
