TLDR: This study evaluated AI-generated high-school physics lesson plans from five large language models (ChatGPT, Claude, Gemini, DeepSeek, Grok) using three prompt frameworks (TAG, RACE, COSTAR). It found that model choice primarily dictates readability, while prompt framework significantly influences factual accuracy and curriculum alignment. Most AI-generated objectives focused on lower cognitive skills (remember and understand). The most effective approach combines a readability-optimized model with the RACE framework and explicit instructions for higher-order objectives and curriculum standards.
Artificial intelligence is rapidly transforming various sectors, and education is no exception. With the advent of powerful generative AI tools, educators are increasingly exploring their potential to streamline tasks like lesson planning. While the convenience of AI-generated lesson plans is undeniable, questions have naturally arisen regarding their quality and suitability for the classroom. A recent study delves into these critical questions, offering valuable insights into how different AI models and prompt structures influence the pedagogical soundness and usability of these plans.
Authored by Xincheng Liu, the research paper, titled “An Evaluation of the Pedagogical Soundness and Usability of AI-Generated Lesson Plans Across Different Models and Prompt Frameworks in High-School Physics,” provides a comprehensive analysis. You can read the full paper here: Research Paper.
Unpacking the Study: Models, Frameworks, and Metrics
The study set out to evaluate lesson plans generated by five prominent large language models: ChatGPT (GPT-5), Claude Sonnet 4.5, Gemini 2.5 Flash, DeepSeek V3.2, and Grok 4. To understand the impact of how instructions are given to these AIs, three structured prompt frameworks were used: TAG (Task, Audience, Goal), RACE (Role, Audience, Context, Execution), and COSTAR (Context, Objective, Style, Tone, Audience, Response Format).
For consistency, all fifteen lesson plans (one for each model-framework combination) were generated for a single high-school physics topic: The Electromagnetic Spectrum. These plans were then rigorously analyzed using four automated computational metrics:
- Readability and linguistic complexity: How easy is the text to understand?
- Factual accuracy and hallucination detection: Is the information correct, and are there any fabricated details?
- Standards and curriculum alignment: Does the plan align with established educational standards like NGSS?
- Cognitive demand of learning objectives: Do the objectives encourage basic recall or higher-order thinking (based on Bloom’s Taxonomy)?
Key Findings: What Matters Most?
The results revealed several fascinating trends:
Readability is Model-Dependent: The choice of AI model had the strongest influence on how easy the lesson plan was to read. DeepSeek consistently produced the most accessible plans, with a Flesch-Kincaid Grade Level (FKGL) of 8.64, which is ideal for high-school students. In contrast, Claude generated the densest language (FKGL = 19.89), making its output more suitable for academic reference than direct classroom use. ChatGPT and Gemini fell in the middle, offering moderate readability.
Prompt Framework Shapes Accuracy and Alignment: While models influenced readability, the prompt framework played a more significant role in factual accuracy and pedagogical completeness. The RACE framework, with its structured components like “Role” and “Execute,” consistently yielded the lowest hallucination index (meaning fewer errors or omissions) and the highest incidental alignment with NGSS curriculum standards. This suggests that guiding the AI with a clear role and execution steps helps it produce more reliable and comprehensive content.
Lower-Order Thinking Dominates Objectives: A consistent finding across all models and frameworks was that the learning objectives primarily clustered at the “Remember” and “Understand” tiers of Bloom’s Taxonomy. This means AI-generated plans tended to focus on basic recall and comprehension rather than encouraging higher-order thinking skills like analysis, evaluation, or creation. This likely stems from the AI models being trained on a vast amount of educational material that often emphasizes these foundational cognitive levels.
Also Read:
- AI Instructor Agents in Higher Education: A Real-World Classroom Experiment
- Interpreting AI’s Math Reasoning: A New Framework for Calculus Education
Practical Takeaways for Educators
The study’s synthesis of these metrics offers clear guidance for educators looking to leverage AI for lesson planning. To create the most effective and usable lesson plans, the research suggests combining a readability-optimized model, such as DeepSeek, with the RACE prompt framework. Furthermore, educators should explicitly include a detailed checklist in their prompts, specifying physics concepts, curriculum standards, and a requirement for higher-order objectives (e.g., at least one “Analyze” and one “Create” level objective) with aligned assessments.
In essence, while AI offers immense potential for efficiency in lesson planning, human guidance remains crucial. By understanding the strengths and weaknesses of different AI models and prompt engineering techniques, teachers can refine their approach to generate AI-assisted lesson plans that are not only convenient but also pedagogically sound, factually accurate, and capable of fostering deeper learning.
