TLDR: This research systematically evaluates the effectiveness of various Explainable AI (xAI) methods and their evaluation metrics for remote sensing (RS) image scene classification. It identifies limitations of existing xAI techniques when applied to RS data, particularly concerning perturbation baselines, multi-label scenes, and large class extents. Through extensive experiments, the study finds that robustness and randomization metrics are most reliable, and GradCAM is the most versatile explanation method for RS, offering practical guidelines for practitioners.
Artificial Intelligence (AI) models are increasingly used in remote sensing (RS) for tasks like classifying land-use and land-cover in satellite images. While these deep learning models are highly accurate, their internal workings often remain a “black box,” making it difficult for humans to understand why a specific prediction was made. This lack of transparency is a growing concern, especially with new regulations requiring interpretability for critical applications.
To address this, Explainable Artificial Intelligence (xAI) has emerged as a crucial field. However, many xAI methods and their evaluation metrics were originally developed for natural images (like photos of everyday objects) and may not be directly suitable for remote sensing images, which have unique characteristics such as multiple spectral bands, multi-label scenes (an image can have several types of land cover), and a lack of clear “background” content.
Investigating XAI in Remote Sensing
A recent study systematically investigated the effectiveness of xAI explanation methods and their evaluation metrics specifically for remote sensing image scene classification. The researchers analyzed ten explanation metrics across five categories: faithfulness, robustness, localization, complexity, and randomization. They applied these metrics to five established feature attribution methods: Occlusion, LIME, GradCAM, LRP, and DeepLIFT, using three different remote sensing datasets.
Challenges with Existing Methods and Metrics
The study identified several key limitations:
Perturbation-based methods (like Occlusion and LIME): These methods work by altering parts of an image to see how the model’s prediction changes. However, defining what constitutes “absence of a feature” in a remote sensing image (e.g., replacing a patch with a black square) can create “Out-of-Distribution” (OoD) samples, which the AI model might not handle reliably. Also, if a land-cover class spans a large, redundant area, perturbing a small part might not significantly change the prediction, making it hard to assess importance.
Gradient-based methods (like GradCAM): These methods use the model’s internal gradients to highlight important regions. They struggle when an image contains multiple instances of the same object or multiple land-cover classes, often localizing only one instance or merging contributions.
Relevancy-propagation methods (like LRP and DeepLIFT): DeepLIFT, similar to perturbation methods, is sensitive to the chosen reference baseline. LRP, which distributes a “relevance score” back to input pixels, can assign disproportionately lower per-pixel relevance to larger land-cover areas compared to smaller ones, even if both contribute equally to the prediction.
Analogously, the evaluation metrics also showed limitations:
Faithfulness metrics: These metrics assess how well an explanation reflects the model’s actual decision-making. They suffer from similar issues as perturbation-based methods, especially concerning the choice of perturbation baseline and the presence of redundant features in RS images.
Localization metrics: These measure how accurately an explanation highlights the correct spatial regions. They rely heavily on precise, pixel-level “ground truth” maps, which are expensive to obtain and can be unreliable. Moreover, they become unreliable when a target class covers a very large portion of the image, as the metric might always yield a perfect score regardless of the explanation’s quality.
Complexity metrics: These quantify the simplicity of an explanation. They often assume that simpler explanations attribute relevance to fewer features. However, in RS, a class might be defined by a larger set of repetitive features across a wide area, which these metrics might incorrectly interpret as high complexity.
Robustness and Randomization metrics: In contrast, these categories of metrics consistently demonstrated greater stability and reliability, being less sensitive to the unique characteristics of RS images.
Also Read:
- Balancing Transparency and Privacy in AI: A Deep Dive into Protecting Explanations
- AI Breakthrough: Enhancing Hurricane Damage Assessment with Multi-Label Classification
Experimental Findings and Recommendations
The researchers conducted extensive experiments on three multi-label remote sensing datasets (BigEarthNet-S2 v2.0, DeepGlobe Multi-Label, and Five Billion Pixel Multi-Label) to validate their methodological findings. They used a framework called MetaQuantus to assess the reliability of the explanation metrics themselves.
The experiments confirmed that robustness and randomization metrics are indeed the most reliable for RS image scene classification. Localization and faithfulness metrics were found to be less reliable, particularly when dealing with the specific challenges of RS data.
When evaluating the feature attribution methods, GradCAM emerged as the most versatile and robust explanation method, consistently performing well across different datasets and evaluation criteria. While no single method was optimal across all criteria, GradCAM showed strong performance in faithfulness, localization, and randomization, making it a suitable general baseline for RS image scene classification.
Based on their comprehensive analysis, the authors provide practical guidelines for practitioners in remote sensing. They recommend prioritizing robustness and randomization metrics due to their proven reliability. For faithfulness metrics, they suggest using a “least relevant features first” removal strategy and a mean-value perturbation baseline to improve reliability. Localization and complexity metrics should be used cautiously, ideally with accurate pixel-level reference maps and an understanding of their limitations when classes span large areas. Finally, GradCAM is recommended as a strong general-purpose explanation method for remote sensing image scene classification. For more in-depth details, you can refer to the full research paper here.
