TLDR: MOIS-SAM2 is a new AI model that improves the interactive segmentation of neurofibromas in whole-body MRI scans. By learning from a few segmented lesions (exemplars), it can efficiently identify many similar lesions across the entire scan, significantly reducing manual effort and outperforming previous methods, even when facing variations in MRI data.
Neurofibromatosis type 1 (NF1) is a common genetic disorder characterized by the growth of numerous benign tumors called neurofibromas (NFs) throughout the body. Monitoring these tumors, especially plexiform neurofibromas, is crucial due to their potential for malignant transformation. Whole-body Magnetic Resonance Imaging (WB-MRI) is the standard clinical tool for detecting and tracking the growth of these lesions over time. However, manually outlining these tumors on MRI scans is an incredibly labor-intensive process, often taking hours per patient, and is prone to variations between different observers.
Existing interactive segmentation methods, which allow users to guide the AI with clicks, face a significant challenge. They either offer broad, scan-wide refinements that lack precision for individual lesions or require a separate user interaction for every single lesion. For NF1 patients who can have hundreds of tumors in a single scan, the latter approach is simply not practical. This highlights a critical need for a solution that can efficiently handle a large number of lesions while still allowing for precise control over individual tumors.
Introducing MOIS-SAM2: A Smarter Approach to Segmentation
A new research paper introduces MOIS-SAM2, a novel multi-object interactive segmentation model designed specifically to address these challenges. MOIS-SAM2 extends the capabilities of the state-of-the-art Segment Anything Model 2 (SAM2) by incorporating an innovative feature called exemplar-based semantic propagation. This means the model learns from a small set of lesions that a user has already segmented and then uses that learned knowledge to automatically identify and segment similar, unprompted lesions across the entire MRI scan.
The core idea behind MOIS-SAM2 is to combine the precision of individual lesion segmentation with the scalability needed for cases with many tumors. When a user segments a few neurofibromas, the model stores these as ‘exemplars’ in an ‘exemplar bank’. A dedicated ‘exemplar attention’ mechanism then uses these exemplars to find other visually and semantically similar lesions throughout the scan, effectively propagating the segmentation knowledge without requiring additional user clicks for every single tumor.
How MOIS-SAM2 Works
MOIS-SAM2 operates in two main stages. First, it performs prompt-based single-lesion instance segmentation, similar to SAM2, where user clicks guide the segmentation of specific lesions. The model also uses a ‘memory bank’ to maintain consistency of these segmentations across adjacent slices. In the second stage, it leverages the ‘exemplar bank’ and ‘exemplar attention’ to perform multi-lesion semantic segmentation. Here, the model identifies all other neurofibromas in the scan that are semantically similar to the previously segmented exemplars, labeling them as foreground without further user input.
This two-stage process allows MOIS-SAM2 to efficiently scale from segmenting a few specific lesions to identifying the entire tumor burden in a WB-MRI scan with minimal user interaction.
Robust Performance Across Diverse Scenarios
The researchers conducted a comprehensive study using 119 WB-MRI scans from 84 NF1 patients. The dataset included various scenarios, such as in-domain scans, scans with different MRI field strengths, cases with low tumor burden, and scans from different clinical sites and scanner vendors. This rigorous testing demonstrated MOIS-SAM2’s superior performance.
In comparison to six other models, including established automated methods like nnU-Net and interactive transformer-based models like SAM2 and VISTA3D, MOIS-SAM2 consistently achieved the highest overall segmentation accuracy and lesion detection rates. Crucially, its performance remained robust even when faced with significant variations in MRI data, a common challenge in real-world clinical settings. The model’s ability to adapt to unseen data characteristics by learning from exemplars within the test scan itself proved vital for this robustness.
Also Read:
- Advancing Medical Image Segmentation with Diffusion Schrödinger Bridge
- Advanced Parkinson’s Diagnosis: Leveraging 2D Vision Models for 3D Brain Scans
Efficiency and Clinical Promise
One of the most significant findings was MOIS-SAM2’s interaction efficiency. It required fewer user clicks to achieve higher segmentation accuracy compared to SAM2, meaning radiologists could segment more lesions with less effort. For instance, performance significantly improved with just two exemplars, and reached a plateau after only 5-6 exemplars, demonstrating its rapid learning capability.
Furthermore, a clinical validity study showed that MOIS-SAM2, even when guided by a non-medical user with a limited number of clicks, achieved segmentation quality comparable to that of expert radiologists. It consistently outperformed less experienced annotators, underscoring its potential to standardize and improve the quality of neurofibroma segmentation in clinical practice.
While the model has some limitations, such as architectural redundancy and the current lack of instance-level separation in its multi-lesion output, its ability to significantly reduce manual effort and provide expert-level segmentation makes it a promising tool. The model and its code are publicly available, and it is designed for integration into clinical workflows via platforms like 3D Slicer, making it accessible for further testing and deployment. This advancement represents a significant step towards more efficient and scalable management of neurofibromas in NF1 patients. You can read the full research paper here.
