TLDR: SAC-MIL (Spatial-Aware Correlated Multiple Instance Learning) is a novel method for Whole Slide Image (WSI) classification in histopathology. It combines the computational efficiency of MLP-based models with the ability of Transformer-based models to capture spatial relationships and full instance correlations. By introducing Polar Rotary Position Embedding (PROPE) for accurate spatial encoding and a specialized SAC block for linear-time instance correlation, SAC-MIL achieves state-of-the-art performance on cancer datasets like CAMELYON-16, TCGA-LUNG, and TCGA-BRAC, while being easy to deploy due to its simple MLP-style architecture.
Whole Slide Imaging (WSI) has become a cornerstone in computer-aided diagnosis, particularly in histopathology, where it plays a critical role in classifying tumors from tissue samples. These images are incredibly high-resolution, often reaching 100,000 x 100,000 pixels, making the manual annotation of tumor regions a time-consuming and challenging task for pathologists.
To address this, researchers commonly employ Multiple Instance Learning (MIL) approaches. In MIL, an entire slide is treated as a ‘bag’ of individual ‘instances’ (patches), and the goal is to classify the whole slide based on these patches, using only slide-level labels rather than detailed patch-level annotations. This significantly reduces the dependency on extensive manual labeling.
Existing MIL methods generally fall into two categories: MLP-based and Transformer-based. MLP-based methods are computationally efficient, processing many instances with linear complexity, which is ideal for the large number of patches in WSIs. However, they often struggle to model the complex correlations between instances and typically ignore the crucial spatial information of each patch. On the other hand, Transformer-based methods excel at capturing these full instance correlations and encoding positional information, but they come with a significant drawback: high computational costs, especially with long sequences, and often require specialized hardware (custom CUDA kernels), limiting their deployment in clinical settings.
Introducing SAC-MIL: Bridging the Gap
A new approach, Spatial-Aware Correlated Multiple Instance Learning (SAC-MIL), aims to combine the best features of both worlds. SAC-MIL is designed to enable MLP-based methods to encode positional information and perform full instance correlations, thereby gaining the advantages of Transformer-based methods without their computational and deployment hurdles. The core of SAC-MIL lies in two innovative components: a positional encoding module and a Spatial-Aware Correlated (SAC) block.
Polar Rotary Position Embedding (PROPE)
Instead of relying on instance indices, which don’t fully reflect the irregular spatial relationships in tissue areas, SAC-MIL introduces Polar Rotary Position Embedding (PROPE). This module encodes the actual spatial coordinates of each instance within the slide. PROPE normalizes the 2D coordinates of both training and testing instances to a consistent range, effectively mitigating the ‘length extrapolation issue’ where training and testing sequences have different lengths. Furthermore, it converts Cartesian coordinates into polar coordinates (radius and angle), ensuring that each dimension captures complete positional information, unlike some 2D positional encoding methods that only encode partial axial information.
The SAC Block: Efficient Instance Correlation
The SAC block is an MLP-based mechanism that performs full instance correlations with linear time complexity. It operates in a two-step process: partial correlation and full correlation. Initially, instances within a region are split into multiple ‘folds’ along the channel dimension. These folds are then shifted with increasing steps, and a channel-wise MLP performs ‘fold-level’ correlations, allowing each fold to encode partial information from other instances in the same region. Subsequently, the folds are shifted back to their original positions, and another channel-wise MLP performs ‘instance-level’ correlations. This second step aggregates information from all folds within the same instance, leading to full instance correlation within each region. The region size expands exponentially with each successive SAC block, ensuring that full instance correlation is achieved across the entire WSI.
Also Read:
- MitoDetect++: Advancing Mitosis Detection and Atypical Subtyping in Pathology with Deep Learning
- Advancing Cancer Diagnosis: YOLOv12 for Automated Mitotic Figure Detection
State-of-the-Art Performance and Practical Advantages
SAC-MIL has demonstrated state-of-the-art performance across various public datasets, including CAMELYON-16, TCGA-LUNG, and TCGA-BRAC. Experiments show significant improvements in metrics like AUC, Accuracy, and F1-score compared to existing MLP-based and Transformer-based methods. For instance, on the CAMELYON-16 dataset, SAC-MIL achieved a 1% performance gain in AUC over the second-best methods using ResNet50 features, and even higher gains with the UNI model as a feature extractor.
Beyond its superior accuracy, SAC-MIL’s simple MLP-style architecture makes it easy to deploy, as it doesn’t require custom CUDA kernels or specific GPU architectures, a crucial advantage in resource-constrained clinical environments. Visualizations of SAC-MIL’s predictions on the CAMELYON-16 dataset further highlight its effectiveness, accurately locating tumor regions with minimal false positives and clear delineation of tumor boundaries.
In conclusion, SAC-MIL represents a significant advancement in WSI classification, offering a robust, efficient, and deployable solution for precise tumor detection. The research paper detailing this innovative approach can be found here.
