TLDR: SpurBreast is a new breast MRI dataset designed to systematically study spurious correlations in deep neural networks (DNNs) used for medical imaging. It intentionally incorporates non-clinical features like magnetic field strength and image orientation that DNNs can exploit, leading to high validation accuracy but poor generalization on unbiased test data. The dataset helps researchers understand and mitigate these biases to develop more robust AI models for breast cancer classification.
Deep Neural Networks (DNNs) have shown incredible promise in medical imaging, often matching or even surpassing human expert performance in diagnosing diseases. However, their real-world application faces a significant hurdle: spurious correlations. These occur when AI models inadvertently learn to associate irrelevant, non-clinical features with disease labels, rather than focusing on the actual medical patterns. This can lead to models that perform well in controlled training environments but fail dramatically when deployed in diverse clinical settings.
Existing medical imaging datasets, while valuable for AI development, were not specifically designed to systematically investigate these spurious correlations. This gap, combined with restrictive licensing and limited patient data, has made it challenging for researchers to understand and address this critical issue. To bridge this gap, a new research initiative introduces SpurBreast, a meticulously curated breast MRI dataset specifically engineered to incorporate and study the impact of spurious correlations on model performance.
The SpurBreast dataset is built upon real-world patient data from the DUKE Breast Cancer Dataset, a comprehensive collection of 3D MRI scans. Beyond the images, it includes extensive supplementary tabular data covering over 100 features related to patient demographics, clinical history, pathology, treatment, imaging devices, and characteristics. This rich metadata allows researchers to identify and intentionally introduce spurious links between these features and the presence or absence of breast tumors.
The methodology behind SpurBreast involves a unique data splitting strategy. Unlike typical random splits, this approach deliberately creates training and validation sets where a specific non-clinical feature is strongly correlated with the tumor label. For example, all tumor-positive images might be associated with one ethnicity, while all tumor-negative images are linked to another. This setup allows researchers to observe if models exploit these artificial shortcuts instead of learning genuine medical indicators.
Through extensive analysis of the supplementary features, the researchers identified two dominant spurious signals that significantly impact model performance: magnetic field strength and image orientation. Magnetic field strength (e.g., 1.5T vs. 3T scanners) is a global feature affecting the entire image, while image orientation (e.g., vertical flipping during data augmentation) is a local feature altering spatial alignment. Other features like ethnicity, menopause status, and surgery type showed weaker, though still present, spurious correlations.
Experimental results using popular deep learning models like ResNet-50 and Vision Transformer (ViT-B/16) on the SpurBreast dataset were striking. While models trained on baseline datasets (without spurious correlations) generalized well, those trained on datasets with strong spurious correlations (magnetic field strength and vertical alignment) achieved near-perfect accuracy during training and validation. However, their performance plummeted to around 50% accuracy on unbiased test data. This dramatic drop clearly demonstrates that the models were heavily relying on these non-clinical attributes as shortcuts, failing to learn meaningful medical features and thus unable to generalize to new, unbiased examples.
For instance, in the case of magnetic field strength, models learned to classify all 1.5T images as tumor-positive and all 3T images as tumor-negative, regardless of actual tumor presence. Similarly, for vertical alignment, images facing one way were consistently predicted as tumor-positive, and the other way as tumor-negative. These findings were consistently replicated across multiple experiments, confirming the robustness of these spurious signals.
Also Read:
- Advancing Breast Cancer Diagnosis: A Transformer-Based Approach
- AI’s Growing Role in Early Osteoporosis Detection and Fracture Risk Assessment
SpurBreast provides a crucial resource for the medical AI community. By offering datasets with both intentionally introduced spurious correlations and unbiased baselines, it enables researchers to systematically investigate how models learn and rely on unintended features. This will facilitate the development of more robust, generalizable, and trustworthy AI models for breast MRI classification, ultimately improving diagnostic accuracy and patient care.
