TLDR: This research introduces an active learning pipeline that combines large foundation models (like CellSAM and microSAM) with traditional neural networks (nnU-Net) to perform biomedical image instance segmentation. The pipeline first uses foundation models to generate pseudo-labels for unlabeled images, enabling nnU-Net’s initial training. Then, a small, representative ‘core-set’ of images is selected and minimally annotated by experts (assisted by microSAM) to fine-tune the nnU-Net. This approach significantly reduces the need for extensive manual annotations, achieving over 90% of full annotation performance with only 6.25-12.5% of labels, making advanced AI techniques more accessible for biomedical researchers.
Biomedical image segmentation is a critical process for precisely outlining structures within images, which is essential for further analysis and quantification. Traditionally, this field has seen significant advancements with deep learning techniques, particularly models like U-Net and its automated successor, nnU-Net. While nnU-Net excels at adapting to various biomedical datasets without extensive manual tuning, it typically demands a substantial amount of annotated data for effective training and cross-validation. This requirement poses a significant challenge, as acquiring labeled data, especially for complex 3D images, is often time-consuming, expensive, and requires specialized expertise.
In parallel, the emergence of large foundation models, such as SAM, SAM2, cellSAM, microSAM, and medSAM, has pushed the field towards more generalist models capable of zero-shot generalizability. However, these powerful models often struggle with the unique characteristics of specific datasets, such as particular tissue environments or drug-treated cells, limiting their direct applicability for precise downstream analysis in specialized biomedical research.
Addressing these critical bottlenecks, a new research paper titled “An Active Learning Pipeline for Biomedical Image Instance Segmentation with Minimal Human Intervention” introduces an innovative data-centric AI workflow. This pipeline strategically combines the strengths of traditional neural networks and large foundation models, aiming to significantly reduce the need for human intervention in the annotation process while maintaining competitive segmentation performance. The full research paper can be accessed here.
The Proposed Active Learning Pipeline
The core of this new approach lies in its two main stages: Pre-training and Fine-tuning. This structured workflow allows nnU-Net to operate effectively even when starting with completely unlabeled raw images.
Stage 1: Pre-training with Pseudo-labels
The pipeline begins by leveraging a large foundation model, specifically CellSAM, to generate “pseudo-labels” for raw, unlabeled biomedical images. These pseudo-labels, though not perfect, provide an initial approximation of the structures to be segmented. This set of raw images and their generated pseudo-labels then forms a preliminary training dataset. This allows nnU-Net to undergo its self-configuration process and initial pre-training, effectively enabling it to learn from unlabeled data without any manual input at this stage.
Stage 2: Fine-tuning with Minimal Human Annotation
Following the pre-training, the pipeline moves to a fine-tuning stage designed to refine the model’s performance with minimal manual effort. This involves selecting a “core-set” – a small, highly representative subset of the unlabeled images. The selection of this core-set is intelligent: raw images are mapped into a latent space using features extracted from a pre-trained Masked Autoencoder (MAE) model. A cosine distance matrix is then computed to identify images that are most diverse and representative of the entire dataset. This ensures that the selected core-set provides maximum information density.
Once the core-set is identified, these selected images are presented to experts for minimal manual annotation. To further reduce human effort, this annotation process is assisted by another foundation model, microSAM. The pre-trained nnU-Net model is then fine-tuned using this small, expertly annotated core-set. This targeted fine-tuning significantly improves the model’s accuracy and precision.
Validation and Key Findings
The effectiveness of this active learning pipeline was rigorously validated using the 3D MitoEM Grand Challenge dataset, which focuses on segmenting mitochondria in electron microscopy images. Crucially, the experiments were conducted under the assumption that no ground truth labels were available for training, mirroring real-world scenarios.
The research revealed several important insights:
-
While pseudo-labels from foundation models are a good starting point, relying solely on them is often insufficient due to inherent inaccuracies like imprecise boundaries and false negatives.
-
The fine-tuning stage with a carefully selected core-set proved highly effective. Remarkably, fine-tuning with just 6.25% (64 samples) of the full manual labels allowed the model to achieve over 90% of the performance seen with a fully labeled dataset for key metrics like F1 Score and Panoptic segmentation. For Accuracy and Precision, 12.5% (128 samples) achieved similar results.
-
The intelligent core-set selection algorithm consistently outperformed random selection methods, demonstrating its ability to identify diverse and information-rich samples for annotation.
-
The combination of pseudo-label pre-training for nnU-Net’s auto-configuration followed by core-set fine-tuning was shown to be the most effective strategy, leading to robust models with significantly reduced manual annotation burden.
Also Read:
- New AI Model JWTH Enhances Biomarker Detection by Fusing Global and Cellular Pathology Insights
- Advancing Cancer Diagnosis: A New End-to-End Approach for Whole Slide Image Analysis
Conclusion
This active learning pipeline represents a significant step forward for biomedical image segmentation. By seamlessly integrating the power of large foundation models for initial pseudo-labeling and assisted annotation with the robust performance of traditional neural networks like nnU-Net, it offers an accessible and efficient solution. This framework addresses the critical need for extensive ground truth data in deep learning and overcomes the limitations of foundation models on highly specific datasets, paving the way for more streamlined and less labor-intensive biomedical research.
