TLDR: Aryabhata 1.0 is a 7B parameter language model optimized for the Indian JEE Math exam. It was developed by merging existing reasoning models, followed by supervised fine-tuning with curriculum learning on verified chain-of-thought traces, and further enhanced with reinforcement learning using verifiable rewards. The model outperforms other LLMs in accuracy and efficiency on JEE Main 2025 and shows strong generalization on MATH and GSM8K benchmarks, providing pedagogically useful step-by-step reasoning.
A new language model named Aryabhata 1.0 has been introduced, specifically designed to help students prepare for the Joint Entrance Examination (JEE) in India. This compact model, with 7 billion parameters, focuses on mathematical reasoning and aims to address the shortcomings of existing large language models (LLMs) in educational settings.
Current LLMs, despite their advancements, often struggle with the specific demands of rigorous academic exams like the JEE. These exams require not just correct answers, but also clear, step-by-step reasoning that students can follow to understand concepts better. Many existing models either provide inaccurate solutions, lack transparent reasoning, or generate explanations that are too long and difficult to follow.
Building Aryabhata 1.0: A Unique Approach
The development of Aryabhata 1.0 involved a multi-stage process to ensure both accuracy and pedagogical utility. The creators, Ritvik Rastogi, Sachin Dharashivkar, and Sandeep Varma, combined several advanced techniques:
First, they used a technique called Model Merging. This involved combining three strong open-source mathematical LLMs: Qwen2.5-Math-7B-Instruct, AceMath-7B-Instruct, and DeepSeek-R1-Distill-Qwen-7B. By merging these models, Aryabhata 1.0 gains a hybrid capability, combining fluent answer generation with coherent, multi-step reasoning.
Next, high-quality, domain-specific data was crucial. The team utilized a proprietary dataset from PhysicsWallah, carefully curated by subject matter experts to align with JEE standards. This dataset, comprising around 130,000 cleaned questions, was filtered to remove diagram-based questions, non-English content, and questions that relied on multiple-choice options.
Following data curation, Supervised Fine-Tuning (SFT) was applied. The model generated multiple chain-of-thought (CoT) responses for each question, and only those that led to the correct answer were selected. This process, called “best-of-n rejection sampling,” ensured the quality of the reasoning traces. The training also used a “curriculum learning” approach, starting with easier problems and gradually introducing more challenging ones.
Finally, Reinforcement Learning with Verifiable Rewards (RLVR) was used to further enhance performance. This stage involved optimizing the model using a binary reward system (correct or incorrect answer) and employing innovative exploration strategies like Adaptive Group Sizing and Progressive Temperature Scaling. These techniques helped the model learn to generate more robust and accurate solutions, especially for difficult problems.
Also Read:
- Unlocking Mathematical Prowess in AI: The Compass-Thinker-7B Model
- Klear-Reasoner: Enhancing AI Reasoning Through Refined Training Methods
Performance and Future Outlook
Aryabhata 1.0 was rigorously evaluated on both in-distribution and out-of-distribution benchmarks. On the JEE Main 2025 exam (in-distribution), Aryabhata 1.0 achieved an impressive 86.0% accuracy in the January session and 90.2% in the April session. It also demonstrated efficiency, generating responses with an average of approximately 2,000 tokens per answer.
For out-of-distribution evaluation, the model was tested on MATH 500 and GSM8K datasets, showing competitive generalization and outperforming its base models. This indicates Aryabhata 1.0’s ability to solve problems beyond its specific training data.
The release of Aryabhata 1.0 marks a significant step towards creating open-source, exam-centric small language models. The developers plan to expand its capabilities to other STEM domains like Physics and Chemistry, cover the full JEE and NEET syllabuses, and continue developing a family of compact, efficient AI tools tailored for Indian education standards. You can find more details about this research in the full paper: Aryabhata: An exam-focused language model for JEE Math.
