TLDR: FairTabGen is a new LLM-based framework for generating synthetic tabular data that unifies counterfactual and causal fairness. It uses prompt engineering and an iterative feedback loop with GPT-4o to create fair and useful data, outperforming existing methods in fairness metrics while retaining utility, even with limited original data.
In today’s data-driven world, generating synthetic data has become a vital tool, especially in situations where privacy is a concern or real data is scarce. This is particularly true for tabular datasets, which are widely used across various real-world applications. However, a significant challenge in creating this artificial data is ensuring it is fair and doesn’t perpetuate or amplify existing biases found in original datasets, while still being useful for machine learning tasks.
Traditional methods for generating synthetic data often focus on statistical accuracy, meaning the generated data looks and behaves like the real data. But they frequently fall short when it comes to fairness, especially when the original data contains structural biases. This is where the concept of “fairness” in data generation becomes critical. The goal is to create data that is not only realistic but also fair, meaning it doesn’t reproduce or worsen social biases.
A new framework called FairTabGen has been introduced to tackle this complex problem. It’s a fairness-aware system that uses large language models (LLMs) to generate synthetic tabular data. What makes FairTabGen unique is its ability to integrate multiple definitions of fairness, specifically “counterfactual fairness” and “causal fairness,” into both its data generation and evaluation processes. This means it aims to ensure fairness at both individual and group levels, considering how different attributes might influence outcomes.
Understanding Fairness in Data
To appreciate FairTabGen’s contribution, it’s helpful to understand the two main types of fairness it addresses. Counterfactual fairness focuses on individual-level non-discrimination. It asks: if only a protected attribute (like race or gender) were different, would the decision or outcome remain the same? Causal fairness, on the other hand, provides group-level guarantees based on explicit structural assumptions. It looks at how bias might enter the system through direct or indirect pathways related to sensitive attributes. FairTabGen argues that both these paradigms are crucial and complement each other.
How FairTabGen Works
FairTabGen operates as a modular and iterative framework. It starts by constructing a “fairness-aware prompt” for the LLM. This prompt includes details like the desired data structure, a small sample of real data for context, and specific fairness and utility constraints. It also clearly identifies sensitive attributes, mediating factors, and outcome variables. The system uses in-context learning, providing the LLM (like GPT-4o) with a few examples from the original dataset to guide its generation, while also encouraging diversity and discouraging simple memorization.
The synthetic data is then generated by GPT-4o, with prompts designed to embed structural dependencies and fairness principles. After generation, each batch of synthetic data undergoes a comprehensive evaluation. This assessment checks for statistical quality (how similar it is to real data), predictive utility (how well models trained on it perform), counterfactual fairness, and causal fairness. If the generated data doesn’t meet the predefined fairness or utility thresholds, FairTabGen enters an “orchestration loop.” In this loop, the system refines the prompts, potentially reweighting group distributions or modifying the prompt structure, until the targets are met. This iterative feedback mechanism is key to achieving both fairness and utility.
Also Read:
- Enhancing LLMs: How Causal Reasoning Reduces Hallucinations
- ToolACE-MT: Crafting Smarter AI Agent Conversations with Non-Autoregressive Generation
Performance and Impact
FairTabGen was tested across three diverse real-world datasets: COMPAS (criminal justice), Law School Admissions (education), and MIMIC-IV (healthcare). These datasets are known for exhibiting real-world fairness challenges. The results were promising: FairTabGen consistently achieved higher fairness metrics, such as demographic parity and path-specific causal effects, compared to existing state-of-the-art methods, including GAN- and other LLM-based approaches. Importantly, it managed to do this while maintaining or even improving the statistical utility and predictive performance of the generated data.
One remarkable finding is its efficiency: FairTabGen achieved these gains using less than 20% of the original data, highlighting its practicality in privacy-sensitive and data-scarce environments. This suggests that LLMs, when guided by structured prompts and fairness-aware feedback, can generate synthetic data that is not only useful but also socially responsible, paving the way for safer deployment of generative AI in sensitive domains.
The framework represents a shift from traditional model design to “prompt orchestration” for fairness-aware synthetic data generation, making the process more interpretable and adaptable. While the current framework has some limitations, such as assuming observed confounders are sufficient and not yet supporting intersectionality, future work aims to evolve it into an AI-agent framework capable of autonomously refining prompts for multi-objective targets.
For more technical details, you can refer to the full research paper: FAIRTABGEN: Unifying Counterfactual and Causal Fairness in Synthetic Tabular Data Generation.
