TLDR: A research paper evaluates AI’s ability to manage a college-level mathematics course across syllabus design, material presentation, student Q&A, and assessment. It finds AI excels in organization and speed but lacks human creativity, emotional intelligence, and nuanced understanding of student needs. The study recommends integrating human expertise with AI’s strengths for better educational outcomes.
A recent research paper delves into a compelling question: Can the current trends of Artificial Intelligence (AI) effectively manage a full course of college-level mathematics? The study, authored by Mariam Alsayyad and Fayadh Kadhem, provides a comprehensive evaluation of AI’s capabilities in four key areas: creating a course syllabus, presenting selected material, answering student questions, and developing assessments.
The researchers aimed to understand the extent to which AI can be reliable in an academic setting and how human and AI strengths can be combined for the best educational outcomes. To achieve this, they conducted a unique comparison: a human-generated version of course materials was created in parallel with an AI-generated version. These materials were then presented to a diverse group of experts, including math professors, high school teachers, graduate students, and professionals in related fields, without revealing whether the content was human or AI-produced. The AI system chosen for this study was ChatGPT, specifically GPT-4o, used in a typical, non-expert manner.
Syllabus Design: AI’s Organizational Edge
When it came to syllabus creation, the study revealed a clear preference for AI in terms of organization. The AI-generated syllabus was rated higher for addressing course aims, having more reasonable learning outcomes, and clearer course descriptions. It also scored better for assessment distribution, appropriateness of assessment types, and overall organization. A notable strength of the AI syllabus was its integration of technology tools like MATLAB, GeoGebra, and Desmos, and its broader coverage of mathematical concepts, aiming for generality across different majors.
However, the human-generated syllabus was preferred for its weekly breakdown of material, weekly workload, and logical sequence. Experts noted that the human syllabus demonstrated experience, was comprehensive, detailed, and aligned with standard textbooks, providing a strong foundation for subsequent math courses. The AI syllabus, while broad, was criticized for being overly ambitious, attempting to cover too many topics superficially without sufficient depth or a single guiding textbook.
Class Presentation: The Human Touch Prevails
In the realm of class presentations, human-generated content significantly outperformed AI across almost all metrics. Human presentations were lauded for their clarity, ease of understanding, relevance, clear objectives, accuracy of mathematical content, comprehensiveness, and smooth flow. Respondents appreciated the gradual progression of ideas, visual aids, worked examples, discussions of common mistakes, real-life applications, and detailed step-by-step solutions that promoted self-learning.
While the AI presentation was praised for its systematic organization and mathematical accuracy, it fell short in providing sufficient examples, diverse exercises, and detailed step-by-step explanations. Suggestions for improving AI presentations included adding more visual representations, a greater variety of challenging examples, real-world applications, and interactive elements.
Answering Student Questions: Empathy and Detail from Humans
The comparison of answers to student questions showed a strong preference for human responses. Out of 16 real student questions, 12 favored human-generated answers. Experts felt that human responses better considered the students’ level, addressed their confusion, and provided detailed, comprehensive, and illustrative explanations. A key observation was the presence of ’emotional intelligence’ in human answers, demonstrating empathy and compassion in clearing doubts, which AI currently lacks.
AI answers, while concise and direct, were often perceived as abstract, too short, and more suitable for experts rather than introductory students. They sometimes lacked critical evaluation and engaging arguments. The paper suggests that while AI can serve as a starting point for generating responses, human refinement is crucial for depth, critical analysis, clear communication, and an empathetic tone.
Assessment Creation: AI’s Clarity vs. Human’s Nuance
For assessment creation, AI showed strengths in clarity. The AI-generated exam was rated higher for the clarity of its question descriptions, making it easier for students to follow. It was also well-aligned with learning outcomes and covered topics comprehensively with questions broken into steps.
However, human-generated assessments were preferred for their ability to differentiate student levels, offering a variety of problems ranging from easy to critical thinking. This allowed for a more comprehensive examination of concepts and tested diverse competencies. The AI assessments were often seen as less challenging and more ‘one-size-fits-all,’ with some respondents even rating its grade distribution as unfair. The study highlights that human instructors, through their interaction with students, gain insights crucial for designing nuanced assessments that AI currently misses.
Also Read:
- Beyond Convenience: How AI Can Truly Boost Our Brainpower Without Making Us Lazy
- Human Bias Undermines AI Oversight: New Research Reveals Challenges
Integrating Human and AI Potential
The research concludes that while AI offers significant advantages in speed, immediate improvements, and organizational capabilities, it still lacks the human creative touch and emotional considerations vital for effective education. AI excels in organizing information and providing accurate answers, making it a powerful tool for educators.
The paper offers several recommendations for leveraging AI in mathematics education: educators should embrace AI for its efficiency and speed, but always provide it with creative ideas, as AI does not generate these independently. AI can be used for general knowledge or a big-picture understanding, but its output must always be cross-referenced with trusted resources and refined by human experts. The ultimate goal is to combine human expertise—in areas like curriculum design, emotional intelligence, and critical thinking—with AI’s technical competencies to create a more effective and enriching learning experience. For more details, you can read the full research paper here.
