TLDR: Dolphin AI introduces Dolphin v1.0 and Dolphin R1, the first large-scale multimodal ultrasound foundation models. These models, trained on a 2-million-scale dataset and a three-stage strategy, unify diverse clinical tasks and achieve state-of-the-art performance on the U2-Bench benchmark, with Dolphin R1 more than doubling the score of its closest competitor. The reasoning-augmented Dolphin R1 significantly enhances diagnostic accuracy and interpretability, marking a major advancement in AI for ultrasound imaging.
Ultrasound imaging is a cornerstone of modern medicine, used widely in fields like obstetrics, cardiology, and emergency care due to its real-time capabilities, portability, and cost-effectiveness. However, integrating artificial intelligence (AI) into ultrasound has been challenging. Issues such as operator dependence, image noise, and the dynamic nature of real-time scanning create unique complexities that traditional large multimodal models often struggle with.
Addressing this critical gap, Dolphin AI has introduced a groundbreaking solution: Dolphin v1.0 (V1) and its advanced version, Dolphin R1. These are the first large-scale multimodal foundation models specifically designed for ultrasound, aiming to unify diverse clinical tasks within a single vision-language framework. This innovation promises to make AI integration in ultrasound more effective and reliable.
A Comprehensive Dataset for Robust Learning
To overcome the inherent variability, noise, and operator dependence in ultrasound imaging, Dolphin AI curated an unprecedented multimodal dataset. This massive dataset, spanning over 2 million samples, combines a rich array of sources: in-depth textbook knowledge, publicly available ultrasound data, synthetically generated knowledge-distilled samples, and general multimodal corpora. This comprehensive approach ensures that the Dolphin models achieve robust perception, strong generalization, and broad clinical adaptability across various medical domains.
The data curation process was meticulous, involving several stages. It included extracting information from classic ultrasound textbooks and guidelines, collecting public datasets for tasks like classification, segmentation, and detection, and integrating general medical data to enhance the model’s overall capabilities. Synthetic data was also generated using question templates, VQA data, and knowledge distillation, all rigorously filtered and validated by medical experts to minimize hallucination and ensure clinical accuracy.
A Progressive Three-Stage Training Strategy
The Dolphin series models are developed using a progressive three-stage training strategy. This approach is designed to integrate domain-specific knowledge, align with human preferences, and refine autonomous decision-making capabilities.
The first stage, Domain-Specialized Training, focuses on injecting ultrasound-specific knowledge into the model while preserving its generalizability. This involves training on extensive textbook-based and public ultrasound data, covering 15 major anatomical systems. The goal is to develop fundamental capabilities in disease diagnosis, anatomical localization, and scan plane recognition.
The second stage, Instruction-Driven Alignment, refines the model’s output to ensure strict adherence to predefined formats and content requirements. This involves fine-tuning with a small-scale instruction dataset derived from distilled knowledge and expert feedback, ensuring consistency with established clinical outputs.
The final stage, Autonomous Reinforcement Refinement, builds upon this foundation by employing reinforcement learning with verifiable ultrasound-specific reward signals. This stage, particularly for Dolphin R1, enables deeper diagnostic inference, enhanced reasoning transparency, and more interpretable decision pathways, crucial for high-stakes medical applications.
Also Read:
- Benchmarking AI in Radiology: A Reality Check on Diagnostic Accuracy
- Assessing Synergy in Unified AI Models: The RealUnify Benchmark
Setting New Benchmarks in Ultrasound Understanding
The performance of the Dolphin models was systematically evaluated using U2-Bench, a comprehensive benchmark designed for eight representative ultrasound tasks, including lesion localization, organ detection, clinical value estimation, and structured report generation.
The results are remarkable: Dolphin R1 achieved a U2-score of 0.5835, more than double the score of the second-best model (0.2968), establishing a new state of the art in multimodal ultrasound understanding. Dolphin v1.0 also delivered competitive performance, validating the effectiveness of the unified training framework. A key finding was that reasoning-enhanced training significantly boosts diagnostic accuracy, consistency, and interpretability, underscoring the importance of integrating reasoning into foundation models for medical domains.
Dolphin R1 particularly excelled in classification and detection tasks, demonstrating strong spatial understanding and anatomical structure recognition. While it showed limitations in clinical value estimation and report generation, its overall performance highlights its robust ability to handle complex ultrasound-specific visual patterns and anatomical variations.
The research also highlighted the significant impact of model scale, with larger 72B parameter models consistently outperforming smaller 7B variants, especially in tasks requiring fine-grained visual features. The deep reasoning mode of Dolphin R1 not only improved quantitative accuracy but also enhanced the interpretability of diagnostic processes, aligning more closely with physician preferences.
This work represents a significant leap forward in ultrasound-based medical AI, paving the way for more accurate, efficient, and intelligent clinical decision-making. For more details, you can refer to the full technical report: Dolphin v1.0 Technical Report.
