TLDR: A new unsupervised, oracle-free framework called IL-PatchCore+SWAG is introduced for incremental anomaly detection in medical imaging. It starts with a small set of normal images and incrementally expands its understanding of “normality” by admitting new unlabeled samples only if they are statistically consistent with existing normal data and the model is highly certain about their normality. This dual-gated approach prevents contamination and significantly improves anomaly detection performance on various medical datasets without requiring any labeled anomalies.
Anomaly detection in medical imaging is a critical task, essential for early diagnosis of conditions like COVID-19 or tumors. However, a major hurdle is the scarcity of labeled anomalous data, which is often expensive, difficult, or even impossible to obtain. Traditional methods like One-Class SVMs and k-Nearest Neighbors struggle with the complexity and high dimensionality of modern medical images, while deep learning approaches like Autoencoders and Generative Adversarial Networks (GANs) can suffer from overfitting to normal data or training instability, limiting their effectiveness in detecting novel anomalies.
Addressing these challenges, researchers have introduced a novel, unsupervised, and oracle-free framework called IL-PatchCore+SWAG. This system is designed to incrementally learn and refine its understanding of what constitutes “normality” in medical images, without ever needing to see a single labeled anomaly. It starts with a small, trusted collection of normal images and gradually expands this set by carefully admitting new, unlabeled samples.
The framework builds upon the efficient PatchCore architecture, which uses pre-trained networks to extract detailed features from images and stores a compact representation of normal patterns in a “memory bank.” To ensure safety during this incremental expansion, the system employs a dual probabilistic gating mechanism. A new sample is only added to the normal memory if two conditions are met: first, its distance to the existing normal patterns is within a statistically calibrated threshold, and second, the model exhibits low “epistemic uncertainty” about the sample’s normality, estimated using Stochastic Weight Averaging-Gaussian (SWAG).
In simpler terms, the system asks two key questions before accepting a new image as “normal”: “Does this look like what I already know to be normal?” and “Am I confident that I know this is normal?” This dual-evidence approach acts as a robust filter, preventing the accidental inclusion of anomalies into the normal dataset, which could otherwise corrupt the model’s understanding of health.
The process involves a frozen pre-trained vision backbone augmented with tiny convolutional adapters. These adapters are lightweight and ensure rapid adaptation to specific medical domains with minimal computational cost. As new, high-confidence normal samples are admitted, these adapters are briefly updated, allowing the model to continuously refine its representations of normality over time. This incremental learning strategy avoids the need for large amounts of labeled data or computationally expensive generative models.
The effectiveness of this framework was rigorously tested on three real-world medical imaging datasets: COVID-CXR, Pneumonia Chest X-ray, and Brain MRI ND-5. The results were substantial. On the COVID-CXR dataset, the ROC-AUC (a measure of diagnostic accuracy) improved significantly from 0.9489 to 0.9982, and the F1 score (a balance of precision and recall) rose from 0.8048 to 0.9746. Similar gains were observed on the Pneumonia-CXR dataset, with ROC-AUC increasing from 0.6834 to 0.8968, and on the Brain MRI ND-5 dataset, where ROC-AUC improved from 0.6041 to 0.7269. These improvements highlight the system’s ability to not only enhance detection sensitivity but also reduce false positives, leading to more reliable and clinically useful anomaly localization.
An ablation study further confirmed that the SWAG-based uncertainty estimation strategy was superior to other methods like MC Dropout and simple ensembles, providing robust and well-calibrated uncertainty estimates crucial for the framework’s safety. While the current evaluation is primarily within the medical domain, the framework offers a practical step towards label-free, uncertainty-aware incremental anomaly detection that could be extended to other fields where anomalies are rare or difficult to label.
Also Read:
- Generative Hints: A Novel Method for Enhancing Model Invariance in Deep Learning
- SPARKER: A New AI Approach for Detecting Subtle Data Anomalies
This research represents a significant advancement for medical diagnostics, offering a scalable and robust solution for identifying unknown anomalies without relying on costly expert supervision or extensive labeled datasets. The source code for this innovative framework is publicly available, fostering further research and development in the field. You can find more details about this work in the full research paper: I Detect What I Don’t Know: Incremental Anomaly Learning with Stochastic Weight Averaging-Gaussian for Oracle-Free Medical Imaging.
