TLDR: This paper reproduces and extends the work of Hu et al. (2024) on Faithful Vision Transformers (FViTs), which use Denoised Diffusion Smoothing (DDS) to improve the robustness and faithfulness of AI model explanations, especially under adversarial attacks. The study confirms that DDS generally enhances interpretability and robustness, particularly with the Attribution Rollout method, but highlights a significant increase in computational cost and environmental impact, questioning its efficiency for large-scale applications.
In the rapidly evolving field of artificial intelligence, understanding why a model makes a particular decision is as crucial as the decision itself. This is especially true for Vision Transformers (ViTs), powerful AI models used in tasks like image recognition and object detection. However, these models often struggle with interpretability – their explanations can become unreliable when faced with even slight changes or attacks on the input images.
A recent study, titled Improving Interpretation Faithfulness for Vision Transformers, delves into this challenge by reproducing and extending the work of Hu et al. (2024). The researchers, Izabela Kurek, Wojciech Trejter, Stipe Frković, and Andro Erdelez from the University of Amsterdam, investigated a method called Denoised Diffusion Smoothing (DDS) which aims to make ViT explanations more ‘faithful’ and robust.
What is a Faithful Vision Transformer (FViT)?
The core idea behind this research is the concept of a Faithful Vision Transformer (FViT). Unlike standard ViTs, FViTs are designed to maintain stable and accurate explanations even when the input images are subtly altered or subjected to adversarial attacks. This stability is achieved through DDS, a novel plug-in method that combines randomized smoothing (adding Gaussian noise) with a diffusion-based denoising model. Essentially, an input image is first noised, then cleaned by a diffusion model, and finally fed into the ViT for analysis. This process helps the model focus on the true features of the image rather than being misled by minor perturbations.
Reproducing and Extending Key Claims
The Amsterdam team set out to verify two main claims made by Hu et al. (2024):
1. **Robust Segmentations**: DDS improves the robustness of image segmentations (highlighting important regions) generated by interpretability methods, particularly under adversarial attacks.
2. **Classification Robustness**: DDS enhances the model’s ability to classify images correctly, even when inputs are perturbed or attacked.
Beyond reproduction, the study also explored new avenues. They investigated whether DDS could improve a newer interpretability method called Attribution Rollout (AR) and if DDS acts as a general ‘plug-in’ to boost the robustness of other common interpretability methods like Raw Attention, GradCAM, and LRP. A critical new aspect of their research was also to measure the environmental impact of using DDS.
Findings: Improved Robustness, but at a Cost
The results largely aligned with the original study’s findings: DDS indeed makes explanations more robust and stable under adversarial attacks. This was evident in both image segmentation and classification tasks. When DDS was applied, the interpretability methods produced more consistent and reliable heat maps, visually demonstrating that the model’s reasoning remained sound despite input perturbations.
Notably, applying DDS to Attribution Rollout (AR) yielded substantial improvements, often achieving the best overall results in both segmentation and classification tasks. For other baseline interpretability methods, DDS generally led to similar or improved metrics, especially for those methods that utilize gradient information, like LRP.
However, the study also uncovered significant discrepancies. The baseline methods (without DDS) often performed worse in their reproduction compared to the original Hu et al. (2024) paper, aligning more closely with earlier work by Chefer et al. (2021). The authors attributed this to not having access to the most up-to-date codebase from Hu et al. (2024).
Perhaps the most striking finding was the environmental impact. The computational cost of applying DDS was found to be substantially higher, increasing runtime and carbon emissions by over ten times in some cases. This raises serious questions about the efficiency of DDS for real-world, large-scale applications, despite its effectiveness in improving interpretability robustness.
Also Read:
- Boosting Video Encoding Efficiency with ResidualViT
- Optimizing AI Training: A Dual Compression Strategy for Vision Transformers in Split Learning
Looking Ahead
While DDS offers a promising path to more reliable AI explanations, its high computational demand is a significant hurdle. Future research, as suggested by the authors, should focus on improving its efficiency. This could involve using lighter-weight denoising models or reducing the number of diffusion steps required, making FViTs more practical for broader adoption without compromising their interpretability and robustness benefits.
