TLDR: A new research paper introduces a method to transfer visual explainability between AI models using ‘task arithmetic’. This technique allows models to gain the ability to explain their predictions without extensive retraining, significantly reducing computational costs. By defining an ‘explainability vector’ from a source domain, the method successfully imparts explanation capabilities to models in target domains, achieving comparable explanation quality to traditional methods like Kernel SHAP with 150 times fewer inferences.
In the rapidly evolving world of artificial intelligence, especially in image classification, models that can not only make predictions but also explain their reasoning are becoming increasingly important. These are known as self-explaining models. While incredibly valuable for applications like medical diagnosis, autonomous driving, and cybersecurity, training these models has traditionally been very expensive, requiring significant data labeling and computational power.
A new study introduces an innovative approach to overcome this challenge by proposing a method to transfer visual explainability from one AI model to another using a concept called ‘task arithmetic’. This technique allows models to efficiently gain the ability to explain their decisions without the need for extensive, costly retraining.
Understanding Task Arithmetic for AI Explainability
Task arithmetic is a powerful framework that treats the ‘knowledge’ or ‘capabilities’ a model learns as mathematical vectors in its parameter space. Imagine a model learning to identify cats (Task A) and another learning to identify cats and explain why it thinks it’s a cat (Task B). The difference between these two models can be represented as a ‘task vector’ that encapsulates the ‘explainability’ skill. This study leverages this idea, defining an ‘explainability vector’ as the difference between a model trained for both prediction and explanation, and one trained only for prediction, both within a ‘source’ domain.
The core idea is based on an analogy: if the relationship between a prediction-only model and a prediction-with-explanation model in a source domain is known, this ‘explainability’ can be transferred to a prediction-only model in a ‘target’ domain. This means a model that has only learned to classify images in a new domain can be given the ability to explain its classifications, without needing new, expensive explanation data for that specific domain.
How the Method Works
The researchers extended existing image classifiers, specifically those based on vision-language models like CLIP (Contrastive Language-Image Pretraining). Their self-explaining model uses a Vision Transformer (ViT) to process image patches and a special ‘domain-specific head’ to predict both class labels and ‘patch-level attributions’—which indicate how much each part of an image contributes to the classification decision. Crucially, the model’s core parameters are reused from pre-trained models, and only the explainability is ‘added’ through task arithmetic.
The process involves training the model on a ‘source dataset’ where both image labels and ground-truth explanations (e.g., generated by methods like Kernel SHAP) are available. Once the explainability vector is derived from this source training, it can then be applied to a model that has only been trained for prediction on a ‘target dataset’ where explanation data is scarce or non-existent. This effectively ‘imparts’ the learned explainability to the target model.
Also Read:
- Smarter Compression: How Explainable AI Makes Deep Neural Networks Smaller and More Accurate
- Evaluating Explainable AI for Remote Sensing Imagery: A Comprehensive Analysis
Promising Results and Efficiency Gains
Experiments across various image classification datasets demonstrated significant success. The method successfully transferred visual explainability from source to target domains, improving explanation quality without sacrificing the model’s ability to accurately classify images. This was particularly effective when transferring between related domains, though the study also explored transfers between less-related ones.
A key finding was the ‘universality’ of explainability learned on large, diverse datasets like ImageNet. The researchers created a dataset called ImageNet+X, which extends ImageNet-1k with explanation supervision. They found that the explainability vector learned from ImageNet+X could improve explanation quality on nine out of ten different target datasets, showcasing its robustness and generalizability.
Perhaps the most impactful result relates to computational efficiency. The proposed method achieves explanation quality comparable to Kernel SHAP, a widely used post-hoc explanation method. However, while Kernel SHAP typically requires around 150 model inferences to generate an explanation for a single image, the new method does it in a single inference. This represents a massive reduction in computational cost, making real-time explanation generation far more feasible.
This research marks a significant step towards making explainable AI more accessible and efficient. By enabling the transfer of visual explainability, it paves the way for deploying self-explaining models more broadly across various applications without the prohibitive training costs previously associated with them. You can read the full research paper for more details at arXiv:2507.04380.
