TLDR: LoRID is a new method that trains smaller AI models to solve math problems by mimicking how humans learn. It uses a “System 1” (intuitive) and “System 2” (knowledge-based) approach, where a large AI model first generates explicit knowledge, and then different parts of the student model interact and self-correct to improve reasoning, leading to significantly better performance on math datasets.
Large Language Models (LLMs) have shown impressive abilities in solving complex mathematical problems. However, these models are incredibly large, requiring massive computational power and making them difficult to use widely. Smaller Language Models (SLMs), while more efficient, often struggle with mathematical reasoning. Current methods to improve SLMs typically involve having LLMs generate vast amounts of data for the SLMs to learn from, a process akin to ‘cramming’ based on experience and intuition, which psychologists call System 1 thinking.
However, human learning isn’t just about intuition. It also involves System 2 thinking, where we first acquire knowledge and then practice applying it to reinforce our understanding. Inspired by these two distinct modes of human thought, researchers have proposed a novel method called LoRID (Reasoning Distillation Method via Multi-LoRA Interaction) to teach SLMs to solve math problems more like human beings.
How LoRID Works: A Four-Stage Process
LoRID breaks down the learning process into four key stages:
1. Knowledge Augmentation: Unlike traditional methods that only generate problem-solving steps, LoRID first prompts a powerful LLM (like GPT-4) to extract and summarize the general knowledge or rules required to solve a specific math problem. This is similar to a teacher explicitly explaining the underlying principles to a student.
2. System 1 Thinking (Intuitive Reasoner – IR): In this stage, a part of the student model is trained to act as an ‘Intuitive Reasoner.’ It directly generates a Chain-of-Thought (step-by-step reasoning) to solve problems, much like how humans might quickly arrive at an answer based on past experience or intuition.
3. System 2 Thinking (Knowledge Generator – KG & Deep Reasoner – DR): To mimic System 2 thinking, LoRID introduces two additional components. The ‘Knowledge Generator’ learns to produce the necessary knowledge when given a problem. Then, the ‘Deep Reasoner’ takes both the problem and the generated knowledge as input to perform more deliberate and structured reasoning. This mirrors how a student first learns a concept and then applies it in practice.
4. Multi-LoRA Interaction: This is where the human-like self-correction comes in. The Intuitive Reasoner and Deep Reasoner, both trained as efficient LoRA blocks on the same student model, compare their outputs. If their answers are inconsistent, they continue to iterate and provide feedback to each other until a consistent solution is reached. This iterative process enhances the SLM’s mathematical reasoning ability through mutual refinement, much like a student re-evaluating their work.
Also Read:
- Adaptive Guidance Boosts Reasoning in Small Language Models
- Beyond Solving: How Large AI Models Learn to Seek Information
Impressive Results and Broader Implications
The LoRID method was tested on two popular mathematical reasoning datasets, GSM8K and MATH, using various base models including LLaMA-2-7B, LLaMA-3-8B, Mistral-7B, Qwen2.5-Math-7B, and DeepSeekMath-7B. The experimental results are highly promising.
LoRID consistently outperformed other state-of-the-art methods that do not use external tools. On the GSM8K dataset, it showed significant accuracy improvements, outperforming the second-best methods by 2.3% to 16.1% across different base models. This demonstrates the effectiveness of integrating System 1 and System 2 thinking. Furthermore, the open-source models trained with LoRID achieved mathematical reasoning capabilities comparable to, and in some cases even surpassing, much larger closed-source LLMs like GPT-4o on certain tasks, despite having far fewer parameters.
The research also highlighted that LoRID’s benefits extend across different levels of problem difficulty and that the method is efficient even with smaller training datasets. A case study illustrated how the knowledge-driven System 2 could correct errors made by the intuitive System 1, showcasing the power of their interaction.
This innovative approach, detailed in the research paper Can Large Models Teach Student Models to Solve Mathematical Problems Like Human Beings? A Reasoning Distillation Method via Multi-LoRA Interaction, marks a significant step towards making advanced AI reasoning more accessible and efficient by drawing inspiration from the very human process of learning and problem-solving.
