TLDR: TableReasoner is a new framework that uses large language models (LLMs) and programming to answer questions from complex tabular data. It tackles challenges like large table sizes and ambiguous information by converting tables into a structured ‘table schema’ and refining it to focus on query-relevant details. The system employs an iterative ‘thought-action-observation’ process and generates executable code to ensure accurate results, particularly for numerical tasks. TableReasoner achieved top results in the SemEval-2025 Task 8 competition, showcasing its effectiveness and scalability for real-world table question answering.
Table Question Answering (TQA) is a specialized area within Question Answering that focuses on extracting information and providing responses to natural language questions using data presented in tables. Unlike tasks involving unstructured text, TQA presents unique challenges due to the inherent characteristics of real-world tabular data. These challenges include the often large size of tables, incomplete semantic information within columns, and potential ambiguity of entities mentioned in the data.
To tackle these complexities, researchers have developed TableReasoner, a novel framework that leverages the power of large language models (LLMs) and a programming-based approach. This system was presented in the research paper titled “TableReasoner: Advancing Table Reasoning Framework with Large Language Models” by Sishi Xiong, Dakai Wang, Yu Zhao, Jie Zhang, Changzai Pan, Haowei He, Xiangyu Li, Wenhan Chang, Zhongjiang He, Shuangyong Song, and Yongxiang Li. You can find more details about their work in the full research paper.
At its core, TableReasoner models a table not as raw text, but using a sophisticated “table schema.” This schema combines both the structural layout and semantic meaning of the table, allowing the system to understand and process even very large tables efficiently. This is a significant improvement over traditional methods that might struggle with the sheer volume of data in large tables due to context length limitations of LLMs.
A key innovation within TableReasoner is its multi-step “schema linking plan.” This process refines the comprehensive global table schema into a more focused version that only retains information directly relevant to the user’s query. By doing so, it effectively reduces ambiguity and helps prevent the LLM from generating incorrect or irrelevant information, a common issue known as hallucination. This focused schema provides precise and sufficient details, which are crucial for refining the query and generating accurate programs.
Furthermore, the reasoning process within TableReasoner is integrated into an iterative thinking architecture. Inspired by the “ReAct” paradigm, this architecture allows for incremental cycles of thinking, reasoning, and reflection. The system continuously evaluates its progress, and if the question isn’t fully answered, it generates new follow-up queries, repeating the process until a satisfactory answer is achieved. This dynamic approach enhances the model’s ability to handle complex reasoning tasks.
The framework also incorporates a programming module that guides the LLM to generate “Program-of-Thoughts” (PoT) solutions. These generated codes are then executed in isolated environments, such as a Python interpreter, to produce verifiable results. This program-assisted solution is particularly effective in mitigating numerical hallucinations that can occur in multi-step data acquisition and processing when relying solely on textual reasoning.
TableReasoner has demonstrated remarkable performance, achieving first place in both subtasks of SemEval-2025 Task 8, a prominent competition for question answering on tabular data. Experiments show that the framework consistently improves accuracy, especially on large-sized tables, and exhibits strong scalability and robustness across various real-world tabular datasets. The system also shows a more balanced performance across different types of questions, including complex list-based queries.
The researchers also explored strategies to further enhance performance, including supervised fine-tuning of the LLMs used in the query refinement and program generation stages, and employing a majority-voting strategy based on self-consistency. These additions further boosted the system’s accuracy.
Also Read:
- Bridging Language and Logic: A New Framework for AI Reasoning
- Advancing Multimodal AI: A New Model for Unified General and Spatial Understanding
While TableReasoner marks a significant advancement in TQA, the authors acknowledge that its iterative nature can be time-consuming due to numerous inference iterations. Future work will explore adaptive action flows to balance inference times with accuracy, and further investigate the impact of different prompt designs on model performance.
