TLDR: TUTORBENCH is a new benchmark and dataset designed to evaluate the tutoring capabilities of large language models (LLMs). It features 1,490 expert-curated samples across six STEM subjects, covering adaptive explanations, feedback, and active learning support, including multimodal inputs. Using an LLM-judge with detailed rubrics, the study found that current frontier LLMs achieve less than 56% accuracy, indicating significant room for improvement in their ability to guide, diagnose, and support students effectively.
As large language models (LLMs) become increasingly common as learning tools, it’s vital to ensure they can effectively handle the complexities of tutoring. This means they need to understand student needs, adapt their guidance, offer personalized support, and maintain accuracy. To address this, researchers have introduced TUTORBENCH, a new dataset and evaluation benchmark designed to thoroughly assess the core tutoring skills of LLMs.
The TUTORBENCH dataset is extensive, comprising 1,490 samples meticulously curated by human experts. These samples are focused on high-school and AP-level curricula across six STEM subjects: Biology, Physics, Chemistry, Statistics, Calculus, and Computer Science. The benchmark covers three common tutoring tasks:
Adaptive Explanation Generation
This task evaluates an LLM’s ability to create explanations that are specifically tailored to a student’s confusion or knowledge gaps. It involves multi-turn interactions where the LLM must identify misconceptions and provide clear, focused responses based on the student’s follow-up questions.
Assessment and Feedback
Here, LLMs are tasked with reviewing a student’s work, identifying mistakes, classifying the type of error (e.g., arithmetic, conceptual), and generating constructive feedback. This simulates real-time evaluation that helps students quickly iterate on their learning.
Also Read:
- New Benchmark Reveals LLM Performance Gaps in Travel Industry Tasks
- Unpacking AI’s Thought Process: A New Framework for Evaluating Tool-Augmented Agents
Active Learning Support
This use case focuses on the LLM’s ability to promote active learning by providing effective hints. When a student is stuck, the LLM should offer just enough information to guide them to the next step without giving away the full answer, thereby preserving student agency.
A key feature of TUTORBENCH is its multimodal design. Recognizing that students often share images of their handwritten work or diagrams, 828 samples in the dataset include images, alongside text-only conversations. This ensures the benchmark reflects real-world usage scenarios.
To evaluate model responses, TUTORBENCH employs a reliable, fine-grained automatic evaluation method using an LLM-judge and sample-specific rubrics. These rubrics, written by human experts, are self-contained and verifiable, capturing the requirements of a desirable response. A weighting scheme is also applied, with critical rubrics carrying higher positive or even negative weights to penalize undesirable behaviors like directly revealing the final answer in active learning scenarios.
The study evaluated 16 frontier LLMs on TUTORBENCH, and the results highlight significant room for improvement. None of the tested LLMs achieved a score greater than 56%. This indicates that current models fall short in exhibiting the full range of tutoring skills needed to effectively guide, diagnose, and support students. All frontier models achieved less than a 60% pass rate on rubric criteria related to these essential skills.
Interestingly, different model families showed varied strengths. For example, Claude models outperformed others in supporting active learning, though they lagged in overall performance and the other two use cases. Gemini 2.5 Pro and GPT-5 achieved the best overall performance, with Gemini 2.5 Pro showing a notable advantage in recognizing student emotions and generating responses with appropriate tone and style. GPT-5 and o3 Pro excelled in factual correctness, student-level calibration, and instruction following.
The benchmark also analyzed performance across different cognitive demands using Bloom’s taxonomy, revealing that models might struggle with recalling and explicitly stating content even while demonstrating advanced reasoning. Furthermore, LLMs performed best in identifying correct or incorrect student steps but struggled to include alternative solutions, examples, or analogies in their responses.
OpenAI’s recently released ‘study mode’ was also evaluated separately. It scored lower than its API-based counterpart, GPT-5, primarily because study mode often provides partial, intermediate responses, expecting user interaction, which doesn’t align with TUTORBENCH’s evaluation of final, complete responses. For more details on the research, you can refer to the original preprint.
By releasing TUTORBENCH, the researchers aim to provide a comprehensive and challenging benchmark to guide the development of the next generation of AI tutors, pushing for models that are truly adept at personalized and effective educational support.
