TLDR: A new deep learning method translates color flow Doppler (CFD) ultrasound videos to greyscale (B-mode) with high realism. The synthetic videos are indistinguishable from real ones to both deep learning models and blinded clinicians, demonstrating foundational capabilities across various anatomies. This technology helps balance and diversify ultrasound datasets, improving the training of AI models for medical diagnosis.
A new deep learning method is making waves in medical imaging, specifically in ultrasound. Researchers have developed a generative model that can translate ultrasound videos from one sub-modality to another, addressing a significant challenge in creating diverse and balanced datasets for AI in medicine.
Ultrasound is a widely used imaging technique globally, crucial for diagnosis across many organ systems. Deep learning has shown great promise in enhancing ultrasound interpretation, from cardiac imaging to fetal screening. However, ultrasound data presents unique difficulties due to its various sub-modalities, such as greyscale (B-mode) and color flow Doppler (CFD). These sub-modalities are often imbalanced in clinical studies, meaning some types of data are much more common than others. This imbalance can hinder the development of robust and generalizable deep learning models.
The presence of CFD signals, for instance, can vary greatly depending on the sonographer, protocol, and the specific anatomical structures being imaged. This can lead to situations where a deep learning model might mistakenly interpret the presence of CFD as a feature of interest, rather than focusing on the underlying anatomy. Removing all CFD images to avoid this issue isn’t ideal, as it discards valuable data and worsens the class imbalance. This is where image translation comes in, offering a way to balance the use of CFD without losing data.
Translating between CFD and B-mode ultrasound is complex. Simple algorithms struggle with varying color palettes, noise profiles, and the need to reconstruct anatomical structures hidden by the CFD signal. The new method presented in this research paper tackles these challenges head-on. It’s a generative approach for ultrasound CFD-greyscale video translation, trained on a massive dataset of 54,975 videos and tested on 8,368. The model uses a combination of pixel-wise, adversarial, and perceptual losses, and employs two networks: one for reconstructing anatomical structures and another for denoising, aiming for realistic ultrasound imaging.
The researchers evaluated their method thoroughly. The synthetic videos generated by the model achieved an average pairwise Structural Similarity Index Measure (SSIM) of 0.91±0.04 when compared to ground truth videos. This indicates a high degree of visual similarity. More importantly, these synthetic videos performed indistinguishably from real ones in benchmark deep learning tasks, such as classification and segmentation. For example, the F1 score for real videos in classification was 0.9, and for synthetic videos, it was 0.89. The Dice score for segmentation between real and synthetic videos was 0.97.
Perhaps the most compelling evidence of the model’s success comes from evaluation by blinded clinical experts. When asked to distinguish between real and synthetic videos, clinicians achieved an overall accuracy of only 54±6%, which is barely better than random chance. This suggests that the synthetic videos are highly realistic and clinically plausible. The model also demonstrated “foundational abilities” by working well on ultrasound images from several clinical domains it was not explicitly trained on, achieving an average SSIM of 0.91±0.05.
This generative translation method not only creates realistic synthetic ultrasound videos but also offers practical utility in improving the yield, diversity, and class balance of datasets for clinical deep learning models. For instance, it helped recover 7.1% more video clips for fetal imaging data used in congenital heart disease detection, significantly improving the representation of under-represented clinical sites and disease lesions.
Also Read:
- AI and Robotics Team Up for Autonomous Cardiac Ultrasound Scanning
- Automating Heart Strain Measurement in TEE with Deep Learning: Introducing autoStrain
While the model shows excellent performance, the researchers acknowledge areas for improvement, such as occasional blurriness and artifacts in synthetic videos, particularly in fetal data which is inherently more challenging. Future work may explore diffusion-based models and integrate CFD region learning directly into the end-to-end pipeline. This research marks a significant step towards expanding the utility of retrospectively collected imaging data and enhancing the toolkit for medical imaging dataset design. You can read the full paper here.
