TLDR: This research paper introduces a new method for evaluating Explainable AI (XAI) tools, focusing on AI models that predict Boolean function outcomes. It proposes a formal measure of variable importance based on ‘actual causality’ and uses Jensen-Shannon divergence for evaluation against a ground truth derived from randomly generated Boolean formulas. The paper also presents B-ReX, a novel black-box XAI tool specifically designed for tabular data and Boolean classifiers. Experimental results show B-ReX significantly outperforms other XAI methods, especially in non-monotonic scenarios, by accurately identifying causal responsibilities without assuming differentiability or monotonicity.
Understanding why an Artificial Intelligence (AI) model makes a particular decision, a field known as Explainable AI (XAI), is crucial for building trust and identifying potential errors. However, evaluating these XAI approaches is notoriously difficult because what constitutes a ‘good’ explanation can be subjective.
A new research paper, titled “Evaluation of Black-Box XAI Approaches for Predictors of Values of Boolean Formulae,” tackles this challenge by focusing on a specific, well-defined problem: AI models that predict the outcomes of Boolean logic functions. This domain is particularly useful because it allows for an unambiguous ‘ground truth’ – a clear, objective measure of what features are truly important for a given outcome.
Previous work in this area attempted to evaluate XAI tools using synthetic datasets based on hand-crafted Boolean formulas. They defined a feature as ‘relevant’ if changing its value would alter the formula’s overall outcome. While informative, this approach had limitations. For instance, in a simple ‘True OR True’ scenario, both inputs contribute to the outcome, but the previous definition might only highlight one as ‘relevant’ because changing either alone wouldn’t flip the result. Also, relying on a small number of hand-crafted formulas could introduce bias.
The authors of this paper, Stav Armoni-Friedmann, Hana Chockler, and David A. Kelly from King’s College London, address these issues by proposing a more formal and precise measure of variable importance based on the concept of ‘actual causality.’ This concept, rooted in the theory of causal models, quantifies the ‘degree of responsibility’ a variable has for an outcome. Instead of simply checking if a feature is decisive, it considers how many other changes are needed to make that feature decisive, offering a more nuanced understanding of importance.
To evaluate XAI tools against this refined ground truth, the researchers moved beyond simple ‘top-k overlap’ metrics. They adopted the Jensen-Shannon (JS-) divergence, a measure that penalizes all mismatches between an XAI tool’s explanation and the true causal responsibilities, not just errors in ranking. Furthermore, to ensure a robust evaluation, they used a large set of randomly generated Boolean formulas, including both monotonic (like AND, OR) and non-monotonic (like NOT, XOR) types, rather than a few hand-crafted ones.
Introducing B-ReX: A Novel Black-Box XAI Tool
A significant contribution of this paper is the introduction of a novel XAI tool called B-ReX. Building upon an existing tool named ReX (which was designed for image classifiers), B-ReX is specifically tailored for explaining tabular data classifiers, particularly those dealing with Boolean formulas. As a ‘black-box’ tool, B-ReX doesn’t need to know the internal workings of the AI model it’s explaining. Instead, it interacts with the model’s interface and observes its behavior.
B-ReX works by iteratively refining its search space to approximate causal responsibility. It intelligently partitions the input data, focusing more closely on areas that appear to have higher responsibility while grouping less important areas. This adaptive partitioning strategy helps it efficiently identify the most influential variables.
Experimental Results and Superior Performance
The researchers conducted extensive experiments, comparing B-ReX against several popular state-of-the-art XAI tools, including ShapleyValues, KernelShap, Saliency, DeepLift, and IntegratedGradients. They trained neural networks to perfectly predict the outcomes of the randomly generated Boolean formulas and then used these XAI tools to explain the network’s decisions.
The results were compelling: B-ReX consistently achieved the lowest Jensen-Shannon divergence from the ground truth across all formula sizes and types. This indicates that B-ReX’s explanations were the most accurate in reflecting the true causal responsibilities of the input variables. Notably, B-ReX demonstrated superior performance, especially when dealing with non-monotonic Boolean formulas (those involving operations like XOR), where other black-box methods often struggled significantly.
The paper explains that traditional methods like Shapley values, derived from cooperative game theory, assume additive contributions, which can be problematic in non-monotonic scenarios. Gradient-based explainers, while performing better than some black-box methods in non-monotonic cases, still rely on differentiability assumptions that don’t perfectly align with the discrete nature of Boolean functions, leading to some overestimation of feature importance.
B-ReX, by contrast, does not assume differentiability or monotonicity of the underlying Boolean formula. This fundamental difference allows it to provide more faithful explanations in complex, non-linear settings. While B-ReX is slower than gradient-based methods due to requiring multiple forward passes, its accuracy in determining causal responsibility is a significant advantage.
Also Read:
- Information Theory Unlocks Deeper Understanding and Diagnosis of Reinforcement Learning Agents
- State Algebra: An Algebraic Framework for Propositional Logic
Limitations and Future Outlook
The study acknowledges its deliberate focus on Boolean formulas, which, while providing a clean benchmark for causal models, limits the immediate generality of the findings to other data types. Additionally, computing the exact ground truth for very large, general Boolean formulas remains computationally intensive, restricting the scale of the evaluation itself.
In conclusion, this research proposes a robust evaluation framework for XAI tools based on the precise measure of causal responsibility. The introduction of B-ReX demonstrates a significant step forward in developing accurate black-box explainers for tabular data, particularly for understanding complex logical relationships. For more details, you can read the full research paper here.
