TLDR: A new research paper introduces a novel partial identification algorithm, IDFair, to assess counterfactual fairness in AI decision-making systems, even when the underlying causal models are unidentifiable. The Bayesian sampling approach derives informative bounds for fairness measures from observational data. Applied to the COMPAS dataset, the study found a significant spurious effect indicating racial bias (African-American individuals more likely to receive higher risk scores) and a negative direct effect for age (older individuals less likely to receive high risk scores), offering a more nuanced understanding of algorithmic discrimination.
The increasing use of Artificial Intelligence (AI) in critical areas like criminal justice, loan approvals, and hiring has brought algorithmic fairness to the forefront of public discussion. A major challenge in evaluating these AI systems is their ‘black box’ nature; we often only see the outputs without understanding their internal decision-making processes. This opacity makes it difficult to assess how decisions might change if sensitive attributes, such as race or age, were different.
Traditional fairness measures often fall short because they assume that true distributions also achieve parity, or they fail to capture indirect discrimination. This has led researchers to explore counterfactual fairness, which asks ‘what if’ questions – for example, ‘what if an applicant’s race were different, while all other factors remained the same?’ If the algorithm’s output changes in such a hypothetical scenario, it suggests unfairness.
However, a significant hurdle in applying counterfactual fairness measures is that they are often ‘unidentifiable.’ This means they cannot be uniquely determined from the available data and qualitative knowledge. A new research paper, titled “PARTIAL IDENTIFICATION APPROACH TO COUNTERFACTUAL FAIRNESS ASSESSMENT” by Saeyoung Rho, Junzhe Zhang, and Elias Bareinboim, tackles this problem head-on. The authors introduce a novel approach using partial identification to derive informative bounds for these elusive counterfactual fairness measures from observational data.
The core of their solution is a Bayesian sampling algorithm, called IDFair, which can bound any counterfactual fairness measure. This algorithm works in several steps: first, it uses a causal discovery algorithm (FCI) to learn potential causal relationships from the data, creating a set of candidate causal diagrams. This set is then refined using existing domain knowledge. For each valid causal diagram, a subroutine called SampleCTF draws posterior samples of the target counterfactual fairness measure, conditioning on the observed data and the inferred causal relationships. Finally, these samples are used to construct a confidence interval, providing a range within which the true fairness measure is likely to fall.
A key innovation of this work is its ability to handle unidentifiable cases without making strong parametric assumptions (like linearity in the model) and its applicability to any categorical data. The algorithm also provides theoretical guarantees for its bounds.
The COMPAS Case Study: Unpacking Algorithmic Bias
To demonstrate their algorithm’s effectiveness, the researchers applied it to the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) dataset. COMPAS is an algorithm used in the judicial process to predict a criminal defendant’s recidivism risk. It gained notoriety in 2016 when ProPublica raised concerns about racial bias, suggesting that black defendants were often predicted to be at a higher risk of recidivism than they actually were, compared to white defendants.
In their analysis, the authors defined protected attributes as Race, Sex, and Age, and other variables like Charge Degree and Prior Counts as potential confounders or mediators. The outcome variable was the COMPAS recidivism risk score. They carefully constructed causal diagrams for each protected attribute, combining insights from the FCI algorithm and domain knowledge.
The findings from the COMPAS case study revealed several important insights:
- Race: For race (African-American vs. Others), the direct and indirect effects were found to be zero, meaning no direct or indirect causal path from race to the COMPAS score. However, the algorithm identified a significant ‘spurious effect’ (SE) with a 95% confidence interval of (0.2348, 0.2771). This suggests that if a non-African-American individual had been African-American, the probability of being assigned a high COMPAS score would have increased by approximately 23% to 27% through ‘backdoor paths’ or unobserved confounding factors. The authors argue that even spurious effects should be considered unfair, especially when the outcome is an algorithmic prediction that might infer sensitive information.
- Age: For age (Less than 30 vs. Over 30), the analysis showed a negative ‘direct effect’ (DE) with a confidence interval of (0.0333, 0.4511). This indicates that being older (over 30) is associated with a lower likelihood of being assigned a high-risk score, directly influencing the COMPAS prediction. The indirect and spurious effects for age were less conclusive due to wider confidence intervals.
- Sex: For sex (Female vs. Male), the direct effect was zero. The indirect and spurious effects had wide confidence intervals that included zero, making it difficult to draw definitive conclusions about discrimination based on sex using these specific measures.
Also Read:
- Unmasking Hidden Bias: A New Framework for Explaining AI Discrimination
- Uncovering Hidden Biases in AI: Introducing humancompatible.detect
Beyond Traditional Measures
The paper also compares its findings with other counterfactual fairness measures like Counterfactual Effect (CE) and Path-Specific Effect (PSE). For instance, with race, CE was inherently zero, failing to capture the significant spurious effect identified by their method. This highlights a crucial point: the choice of fairness metric can lead to vastly different conclusions, and the family of effects (Direct, Indirect, Spurious) used in this paper offers a more granular understanding of algorithmic bias.
This research provides a robust framework for assessing algorithmic fairness, particularly in scenarios where full causal models are unknown and traditional identification methods fall short. By providing informative bounds, it equips auditors and developers with a powerful tool to understand and mitigate bias in AI decision-making systems. You can read the full research paper here: Partial Identification Approach to Counterfactual Fairness Assessment.
