TLDR: The GeoThought research introduces a novel dataset, GeoThoughts (Geo-Thought-6K and Geo-Thought-Augmented-10K), specifically designed to enhance mathematical geometry reasoning in Vision-Language Models. Current AI models struggle with visual geometry due to its complexity and a lack of detailed reasoning data. GeoThoughts addresses this by providing comprehensive entries with visual descriptions, step-by-step solutions, explicit Chain-of-Thought (CoT) reasoning, and reflection steps. Models trained on GeoThoughts, such as GeoThought-MLLM, demonstrate substantial performance improvements on geometric tasks, outperforming existing benchmarks and showing strong generalization. The study also identifies spatial misjudgment as a primary source of errors, which CoT corrections can effectively resolve, highlighting future research directions in visual perception for AI.
Large language models (LLMs) have shown impressive abilities in solving text-based math problems. However, when these models are asked to tackle visual reasoning tasks, especially those involving geometry, their performance often drops significantly. This decline is due to two main challenges: the inherent complexity of geometry, which requires detailed image understanding and multi-step thinking, and the limitations of current datasets, which often lack the necessary scale, diversity, and explicit reasoning steps needed for effective model training.
To address these critical issues, researchers have developed a new dataset called GeoThoughts. This comprehensive collection of geometric reasoning problems includes two main subsets: Geo-Thought-6K, with 6,243 samples, and its expanded version, Geo-Thought-Augmented-10K, containing 10,834 samples. Each entry in the GeoThoughts dataset is rich with information, featuring visual descriptions, detailed step-by-step solutions, clear reasoning chains, reflection steps, and the final answers. This structured approach is designed to guide models through complex problem-solving processes.
The concept of Chain-of-Thought (CoT) reasoning is central to GeoThoughts. CoT helps models by breaking down complex problems into a series of intermediate reasoning steps, which not only improves accuracy but also makes the reasoning process more akin to human cognitive patterns. In geometry, CoT allows models to systematically analyze relationships, form hypotheses, decompose problems, and perform self-verification.
The creation of the GeoThoughts dataset involved an innovative data generation pipeline. It started by using problems and images from existing datasets to query advanced ‘teacher models’ like Doubao-1.5-thinking-vision-pro. These teacher models generated complete CoT sequences and answers. These initial problem-image pairs then served as a foundation to synthesize five additional problems using sophisticated prompt engineering. To ensure high quality, each newly generated problem was queried eight times, and only instances with 100% consensus among the responses were retained, guaranteeing data reliability.
Using this meticulously constructed dataset, the researchers developed GeoThought-MLLM, a mathematical reasoning multimodal model. This model is specifically designed to generate detailed thinking processes as it solves geometric problems. Extensive experiments have shown that GeoThought-MLLM significantly outperforms existing benchmarks in geometric tasks. For instance, models like InternVL3-8B saw performance improvements from 51.19% to 73.21% on GeoQA benchmarks when trained with the GeoThought-Augmented-10K dataset. These consistent gains across different model architectures highlight the superior cognitive advantages of CoT generation in geometric reasoning, both for problems within its training domain and for entirely new, out-of-domain problems.
The research also includes an in-depth analysis of GeoThought-MLLM’s performance, particularly focusing on its failure cases. It was observed that errors primarily arise from incorrect interpretation of mathematical concepts or spatial misjudgment within geometric figures. For example, models might misidentify consecutive interior angles as corresponding angles. Interestingly, when these mistakes in the reasoning chain were manually corrected, the model was able to produce the correct answers, suggesting that accurate spatial judgment is a key area for future improvement. Potential solutions to mitigate these ‘visual forgetting’ issues include re-injecting visual information at various reasoning points or using agent frameworks for multi-turn interaction with images.
Also Read:
- Evaluating VLM Spatial Reasoning: A New Dynamic Benchmark for Solid Geometry
- AI’s Journey to Solve Math Word Problems: A Cognitive Perspective
The GeoThoughts dataset and the GeoThought-MLLM represent a significant step forward in enhancing mathematical geometry reasoning in AI. By providing high-quality, detailed reasoning patterns, this work confirms the efficacy of explicit Chain-of-Thought training and paves the way for more robust and human-like geometric problem-solving capabilities in multimodal AI systems. You can find the full research paper here: GeoThought Research Paper.
