TLDR: LeapFactual is a new algorithm that generates reliable and informative visual counterfactual explanations for machine learning models. It addresses limitations of existing methods by using conditional flow matching, allowing it to work even when model and true decision boundaries diverge, and with non-differentiable systems. Experiments show it produces accurate, in-distribution explanations that can even improve model performance, expanding AI interpretability to diverse applications like scientific research and citizen science.
Understanding why a machine learning model makes a particular decision is becoming increasingly important, especially as these models are used in critical fields like healthcare and scientific research. One powerful way to gain this understanding is through ‘counterfactual explanations’. Imagine a model predicts you have a certain condition; a counterfactual explanation would tell you the smallest changes needed in your input data (e.g., medical readings) for the model to predict you don’t have that condition. This offers deep insights into how the model works.
However, current methods for generating these explanations face several challenges. They can struggle with issues like ‘gradient vanishing’ (where the model’s internal signals become too weak to guide changes), ‘discontinuous latent spaces’ (where the model’s internal representation of data has gaps, making smooth changes difficult), and an over-reliance on how well the model’s learned decision boundaries match the true ones. These limitations often lead to explanations that are either unrealistic or not very helpful.
Introducing LeapFactual: A New Approach to Visual Explanations
To overcome these hurdles, researchers have developed a novel algorithm called LEAPFACTUAL. This new method is designed to generate reliable and informative visual counterfactual explanations, even when there’s a mismatch between what the model has learned and the actual underlying decision rules. A key advantage of LEAPFACTUAL is its ‘model-agnostic’ nature, meaning it isn’t limited to models with easily differentiable loss functions. It can even work with ‘human-in-the-loop’ systems, where human annotators are part of the decision-making process, expanding its use to areas like citizen science.
LEAPFACTUAL achieves its capabilities by using a technique called ‘conditional flow matching’. It cleverly combines the strengths of two existing types of counterfactual generation methods—Optimization-based and Conditional Generative Model-based approaches—while avoiding their common pitfalls. It introduces a unique way of representing data internally, bridging what are called ‘flattened’ and ‘structured’ latent spaces. Think of it as having a flexible pathway that allows the model to smoothly transform an input to show what would have been different.
How LeapFactual Works: Blending and Injection
The core of LEAPFACTUAL involves two main processes: ‘lifting’ and ‘landing’. Lifting effectively removes class-related information from a data point, while landing reintroduces it, but for a different target class. This allows for two powerful explanation strategies:
-
Information Blending: This generates ‘local counterfactuals’ that preserve the original data’s unique characteristics while smoothly transitioning it to a new class. For example, if you want to see how a handwritten ‘8’ could become a ‘6’, blending would show you an ‘8’ that gradually takes on the features of a ‘6’ without losing its original handwriting style.
-
Information Injection: This optional step refines the counterfactual by explicitly injecting target class information. This is crucial because simply reaching the model’s learned decision boundary might not be enough; the explanation needs to align with the true characteristics of the target class. This ensures the generated counterfactuals are not just classified differently, but also genuinely look like typical examples of the new class.
Real-World Applications and Benefits
The effectiveness of LEAPFACTUAL has been demonstrated through extensive experiments on various datasets. On the Morpho-MNIST dataset, which is designed to test how well models learn representations, LEAPFACTUAL consistently outperformed existing methods in both generating correct counterfactuals and maintaining similarity to the original input. The version with information injection, LEAPFACTUAL_R, showed even greater accuracy.
Perhaps one of the most exciting findings comes from the Galaxy10 DECaLS dataset, used for classifying galaxy shapes. Researchers found that reliable counterfactuals generated by LEAPFACTUAL_R could actually be used as new training data to improve the performance of a ‘weak’ classifier. This suggests that these reliable explanations align more closely with the true underlying decision boundaries, making them valuable for data augmentation, especially for small or imbalanced datasets. This could help address issues of fairness and underrepresentation in machine learning models.
Furthermore, LEAPFACTUAL’s generalization capabilities were tested on the FFHQ dataset, which contains high-quality human faces. Using a pre-trained CLIP model as a stand-in for human judgment (a non-differentiable classifier), LEAPFACTUAL successfully transformed faces from a ‘Face’ category to a ‘Smiling Face’ category and vice-versa. This highlights its scalability and applicability to complex, real-world scenarios where traditional methods would fail due to their reliance on differentiability.
Also Read:
- Unlocking Galaxy Classification: Explaining AI Decisions with Counterfactuals
- Enhancing Trust in AI: A New Framework for Explaining Decisions and Detecting Bias
Looking Ahead
LEAPFACTUAL represents a significant step forward in creating interpretable and reliable visual counterfactual explanations. By offering a flexible mechanism to understand model decisions and providing actionable insights, it promotes transparency and broadens the applicability of AI in high-stakes domains. While the current focus is on visual data, the method is broadly applicable to other data types. Future work aims to improve its efficiency in very high-dimensional latent spaces and explore its potential for addressing latent space entanglement and challenges with imbalanced datasets. You can read the full research paper here: LeapFactual: Reliable Visual Counterfactual Explanation Using Conditional Flow Matching.
