TLDR: This research introduces an efficient prompt-based continual learning (PCL) method for medical AI, addressing challenges like data sharing restrictions and catastrophic forgetting. By using a unified prompt pool, minimal prompt expansion, and a novel regularization term with a strong backbone (Dino-v2), the approach significantly improves classification accuracy and F1-score on diabetic retinopathy datasets while reducing computational costs, paving the way for sustainable distributed medical AI.
Modern Artificial Intelligence (AI) models often require vast amounts of high-quality data to achieve their best performance. However, in the medical field, strict ethical, social, and institutional rules severely limit how data can be shared. This makes it almost impossible to train AI models in a single, centralized location. Instead, each hospital or institution must update its AI models incrementally, using only the data it has locally.
This incremental updating presents a significant challenge: traditional AI training methods are prone to ‘catastrophic forgetting.’ This means that when a model learns new information, it tends to forget previously acquired knowledge. Additionally, medical data can vary significantly due to different diagnostic equipment and patient demographics across hospitals, leading to shifts in data distribution that can degrade model performance.
To address these critical issues, researchers have proposed a novel approach called Prompt-based Continual Learning (PCL). This method is specifically designed for the medical domain and focuses on computational efficiency and data privacy. The core of their innovation lies in a ‘unified prompt pool’ combined with a ‘minimal expansion strategy’ and a new ‘regularization term’.
How the New Approach Works
The proposed PCL method aims to enable AI models to continuously learn from new data generated at different hospitals without forgetting what they’ve learned before. Here’s a breakdown of its key components:
-
Unified Prompt Pool: Unlike previous prompt-based methods that might use separate prompt sets for different layers of the AI model, this approach consolidates them into a single, shared pool. This allows the model to fine-tune its learning to capture subtle, yet crucial, distinctions in medical images, which are often acquired under standardized protocols.
-
Minimal Prompt Expansion: To keep computational costs low and prevent the model from becoming too complex, only a small subset of new prompts (about 20% of the original set) is added at each new learning stage. All previously learned prompts are kept fixed, ensuring that the model retains its existing knowledge while efficiently adapting to new information.
-
Novel Regularization Term: A new mathematical component is added to the model’s learning process. This term helps ensure that the newly introduced prompts effectively capture new information without overlapping with or causing the model to forget existing knowledge. It helps balance the need to retain old information with the need to adapt to new data.
-
Strong Backbone Model: The researchers utilized a powerful pre-trained model called Dino-v2 as the foundation for their AI system. This model, trained on massive datasets, is excellent at extracting high-quality features, which is particularly beneficial for handling the severe class imbalances and limited sample sizes often found in medical datasets.
Also Read:
- RETFound Foundation Model Adapted for Precise Optic Disc Segmentation
- Unlocking Efficient Data Learning with Compressive Meta-Learning
Experimental Results and Impact
The effectiveness of this new PCL approach was tested on three public datasets related to diabetic retinopathy: Aptos2019, LI2019 (DDR), and Diabetic Retinopathy Detection. The results were highly promising:
-
The model significantly improved final classification accuracy by at least 10% and the F1-score by 9 points compared to existing state-of-the-art methods.
-
Despite its superior performance, the new method also lowered inference costs, as it uses a single AI backbone model compared to some existing approaches that require multiple backbones.
-
Analysis showed that the prompts learned independently at each stage, effectively encoding novel information without redundancy, and focusing on subtle distinctions crucial for medical imaging.
This study marks a significant step towards developing sustainable medical AI. By enabling models to incrementally update using local data while preventing catastrophic forgetting and managing domain shifts, it paves the way for real-time diagnosis, continuous patient monitoring, and advanced telemedicine applications in distributed healthcare settings. The code for this research will be made publicly available upon acceptance of the paper, fostering further advancements in the field. You can read the full research paper here: Towards Efficient Prompt-based Continual Learning in Distributed Medical AI.
While the current research focused on diabetic retinopathy, future work aims to expand the evaluation to other medical imaging modalities like CT and MRI, different task types such as segmentation and detection, and to explore adaptive prompt expansion strategies and other medical-specific AI models.
