TLDR: This research introduces a novel framework using Subjective Logic to assess the trustworthiness of AI training datasets, focusing on properties like bias that emerge at the dataset level. It proposes a formal methodology for quantifying trust opinions, even with incomplete or conflicting evidence. The framework was experimentally validated on a traffic sign recognition dataset, demonstrating its ability to detect class imbalance and its applicability in both centralized and federated learning environments, contributing to more transparent and fair AI systems.
As artificial intelligence (AI) systems become increasingly integrated into critical areas like healthcare and autonomous vehicles, the reliability of the data they learn from is paramount. While much attention has been paid to the trustworthiness of AI models themselves, the underlying training datasets have often been overlooked. Yet, issues within these datasets, such as sampling bias or label noise, can significantly undermine a model’s fairness, robustness, and interpretability.
Addressing Dataset Trustworthiness
A new research paper, titled “Assessing Trustworthiness of AI Training Dataset using Subjective Logic – A Use Case on Bias”, introduces a groundbreaking framework to formally assess the trustworthiness of AI training datasets. Authored by K. Ismael Ouattara, Ioannis Krontiris, Theo Dimitrakos, and Frank Kargl, this work is particularly significant because it focuses on properties that emerge at the dataset level, such as bias, rather than just individual data points. This approach allows for evaluations that account for uncertainty, especially when evidence is incomplete, distributed, or conflicting.
The core of this framework is built upon Subjective Logic, a mathematical framework designed for reasoning under uncertainty. Subjective Logic allows for the expression of ‘opinions’ about propositions, incorporating belief, disbelief, and uncertainty based on available evidence. This makes it an ideal tool for evaluating dataset trustworthiness, where information might be fragmented or ambiguous.
A Structured Approach to Trust Assessment
The methodology proposed in the paper involves several key steps:
- Defining the specific trustworthiness property to be assessed (e.g., “the dataset is not biased”).
- Breaking down this property into smaller, quantifiable atomic propositions.
- Identifying ‘trust sources’ – entities or mechanisms that provide evidence for these propositions.
- Collecting evidence from these sources.
- Quantifying the trust opinion using one of three methods: Baseline-Prior, Constant-Uncertainty, or Evidence-Weighted Quantification, each suited for different evidence characteristics.
- Combining these opinions using logical operators to arrive at a final assessment.
Focusing on Bias: A Practical Application
The researchers specifically applied their framework to the property of dataset bias. They define bias as a systematic deviation in data representation that can lead to unfair or skewed model behavior. In their framework, the proposition “the dataset is not biased” is broken down into two atomic propositions: “the dataset is accurate” and “the dataset is representative.”
To quantify bias, the paper introduces two primary methods:
- Method 1: Class Probabilities-Based Quantification: This method assesses how well a dataset represents different classes by analyzing the distribution of class probabilities. It’s particularly useful for understanding uncertainty in class assignments and is well-suited for centralized datasets where the total dataset size influences uncertainty.
- Method 2: Entropy-Based Quantification: This method uses entropy, a measure of uncertainty in a probability distribution, to assess class balance within sub-datasets. It’s ideal for scenarios like federated learning, where data comes from multiple, potentially imbalanced sources, and it can operate without needing access to raw labels, enhancing privacy.
Also Read:
- Building Trust in AI: New Criteria for Robust Explanations
- A Unified Framework for Understanding Trust in Learning and Belief Updating
Experimental Validation with Traffic Signs
To demonstrate the framework’s effectiveness, the authors conducted experiments using the German Traffic Signs Dataset. They explored two scenarios: a centralized dataset (original vs. augmented for balance) and a collaborative dataset contributed by multiple manufacturers (OEMs), some of which were intentionally imbalanced.
The results showed that the framework successfully identified class imbalances. In the centralized scenario, data augmentation improved class balance and reduced bias, which was reflected in higher trust opinions. In the collaborative scenario, the negative impact of imbalanced sub-datasets on model accuracy was clearly observed. The two quantification methods behaved distinctly: Method 1’s uncertainty remained stable, reflecting its reliance on total dataset size, while Method 2’s uncertainty gradually decreased as more sources contributed, making it suitable for dynamic, distributed environments.
This research marks a significant step towards more transparent and fair AI systems by providing a quantifiable way to assess the trustworthiness of their foundational training data. For more details, you can access the full paper here.
