TLDR: LPPO is a novel framework that improves Large Language Model (LLM) reasoning by shifting focus from data volume to individual sample learning. It employs Prefix-Guided Sampling to provide partial solution hints for challenging problems and Learning-Progress Weighting to dynamically prioritize samples where the model is actively improving. This sample-centric approach leads to faster training, higher accuracy, and better generalization on mathematical reasoning tasks, even with small, high-quality datasets, and demonstrates robustness across various model scales and architectures.
Large Language Models (LLMs) have made incredible strides in complex reasoning, especially with the help of Reinforcement Learning with Verifiable Rewards (RLVR). While much of the focus in this area has been on designing better algorithms or curating vast amounts of data, a new research paper introduces a fresh perspective: focusing on individual training samples rather than just the sheer volume of data.
The paper, titled “From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization,” highlights a critical challenge: high-quality reasoning data is often scarce and expensive to acquire. Instead of constantly seeking more data, the authors ask how we can make the most of a small, trusted set of high-quality examples, especially when an LLM struggles with a particular problem.
To address this, the researchers from Zhejiang University and Alibaba Group Tongyi Lab propose a novel framework called LPPO, which stands for Learning-Progress and Prefix-guided Optimization. LPPO shifts the paradigm from a purely data-centric approach to a more sample-centric one, dynamically adjusting how the model learns from each example throughout its training.
How LPPO Works: Two Key Strategies
LPPO integrates two complementary techniques inspired by how humans learn:
1. Prefix-Guided Sampling (PG-Sampling): Imagine a student struggling with a math problem. A teacher might offer a hint – a partial solution – to guide them without giving away the entire answer. PG-Sampling works similarly for LLMs. For challenging problems that the model fails to solve, it provides a partial solution prefix from an expert-generated answer. This acts as a hint, guiding the model’s exploration and helping it complete the solution. This method is an online data augmentation technique, meaning it generates these hints during the training process itself, focusing on the problems where the model needs the most help.
2. Learning-Progress Weighting (LP-Weighting): Just as humans naturally focus more on questions where they are actively improving, LP-Weighting dynamically adjusts the importance of each training sample. It tracks the model’s progress on each individual sample over time. If the model is showing significant improvement on a particular problem, that sample’s influence on the training process is amplified. Conversely, if learning has stalled or degraded on a sample, its influence is reduced. This ensures that computational resources are efficiently allocated to samples that are actively fostering learning, accelerating convergence and improving overall efficiency.
The LPPO framework also incorporates an online data curation strategy. This means that samples that become too easy (100% pass rate) or remain consistently too difficult (0% pass rate without guidance) are dynamically excluded from the current training batch. This allows the model to concentrate its efforts on the most informative examples.
Also Read:
- ReMix: Supercharging LLM Reasoning with Smart Data Reuse
- FR3E: Enhancing LLM Reasoning Through Structured Exploration
Impressive Results on Mathematical Reasoning
The researchers tested LPPO on mathematical reasoning benchmarks using the Qwen2.5-Math-7B model. Their experiments showed that LPPO significantly outperforms strong baseline models. The combination of LP-Weighting and PG-Sampling led to a substantial increase in performance, achieving higher average scores across various benchmarks like AIME24, AIME25, and Minerva. The framework also demonstrated faster convergence during training and better generalization capabilities.
Furthermore, LPPO proved to be robust across different scenarios. It consistently improved performance even when applied to larger models (Qwen-2.5-14B), different model architectures (Llama-3.2-3B-Instruct), and alternative reinforcement learning algorithms (REINFORCE++), all without needing extensive hyper-parameter re-tuning. This suggests that the sample-centric approach is broadly applicable and effective.
In conclusion, LPPO offers a practical and effective way to enhance LLM reasoning, particularly when high-quality data is limited. By intelligently focusing on individual learning dynamics and providing targeted guidance, this framework helps LLMs learn more efficiently and achieve higher accuracy in complex tasks. You can read the full research paper at arXiv:2507.06573.
