TLDR: A pilot study explored how undergraduate students in proof-based math courses (abstract algebra, topology) used and perceived generative AI (Microsoft CoPilot) under a policy allowing its use but requiring critical verification. Findings show students primarily used AI for information, brainstorming, and concept explanation, rather than direct proof generation. While generally finding AI helpful for learning, students remained critical of its accuracy in proofs and expressed mixed feelings about its impact on independent problem-solving and peer interaction. The study highlights the need for guided AI integration that fosters critical thinking and verification skills.
Generative Artificial Intelligence (AI) is rapidly changing the landscape of higher education, and its impact on specialized fields like proof-based mathematics is a subject of growing interest. A recent pilot study, titled “Gen AI in Proof-Based Math Courses: A Pilot Study”, delves into how students in advanced undergraduate mathematics courses interact with and perceive generative AI tools. Conducted by Hannah Klawa, Shraddha Rajpal, and Cigole Thomas, this research offers valuable insights into the opportunities and challenges of integrating AI into rigorous academic settings.
The study focused on three proof-based undergraduate mathematics courses: a first-semester abstract algebra course, a topology course, and a second-semester abstract algebra course. A key aspect of this research was the implementation of a course policy that permitted the use of generative AI, specifically Microsoft CoPilot, but required careful referencing of textbook theorems and definitions. This approach aimed to encourage critical engagement rather than passive acceptance of AI-generated content.
Student Engagement with AI
The findings revealed a nuanced picture of student interaction with AI. While 8 out of 19 participants reported not using generative AI for their coursework, a notable 7 participants viewed its use as a form of cheating, even though it was permitted. For those who did use AI, the primary applications were for brainstorming, seeking feedback, explaining complex concepts, and finding references for further learning. Students found it particularly useful for information retrieval, often describing it as a “complex dictionary” or a tool to “go deeper into some of the history and the concepts.” Less frequently, it was used for directly solving questions or checking proofs, with students often seeking starting points rather than complete solutions.
Perceived Helpfulness and Limitations
Overall, students generally found Microsoft CoPilot to be a helpful tool for learning in proof-based mathematics. Many appreciated its ability to aid in understanding material and brainstorming problem-solving approaches. One student enthusiastically called it a “godsend,” noting that the fact it wasn’t always right was helpful as it forced them to “think through the problem, not just accept the answer.” This highlights a potential benefit: using AI to identify errors can become a form of “debugging code,” enhancing critical thinking skills. However, about half of the students did not support incorporating generative AI across all mathematics courses, suggesting a preference for selective use. There were also mixed feelings about its impact on independent problem-solving abilities, with many acknowledging that Copilot struggled with complex problems or providing complete proofs.
Impact on Engagement and Reliability
Interestingly, the study found that using Copilot did not generally reduce students’ engagement with instructors. However, its effect on peer interaction was more varied. Some students felt that AI use elevated the quality of discussions by helping peers recall definitions or identify knowledge gaps, leading to deeper conversations. Conversely, others felt a loss of human connection, suspecting that some discussion posts were “entirely AI generated” and expressing frustration at feeling like they were talking to “unfeeling robots.”
Regarding reliability, students approached AI-generated content with caution and discernment. Many regularly identified inaccuracies in the tool’s responses and were vigilant in verifying information. This skepticism was reinforced by the course policy requiring verification, underscoring the importance of critical engagement in proof-based disciplines where precision is paramount. Students often found AI useful for collecting information but noted its tendency to present inaccurate information with a confident tone, especially for advanced proof-based concepts.
Also Read:
- Navigating the Canvas: How Young Artists Co-Create with Generative AI
- Voice-Enabled AI Tutors in Programming: A Study on Novice Learners
Ease of Use and Future Directions
Most students found Microsoft CoPilot intuitive and easy to use, with minimal challenges in learning its core functionality. However, some suggested that an instructor demonstration or overview of effective AI utilization would be beneficial. The course policy’s requirement for precise citations to textbook theorems and page numbers, while adding some “tedium,” also encouraged careful engagement with the material and reinforced attention to detail.
The study concludes that generative AI can serve as a valuable supplementary learning aid in advanced mathematics education, particularly when students are guided to use it critically. It can clarify concepts, generate ideas, and organize problem-solving strategies. However, concerns about accuracy, trust, and the potential erosion of independent reasoning remain. Future considerations include explicit modeling by instructors on how to productively and responsibly integrate AI, including prompt phrasing, critical evaluation of outputs, error identification, and proper citation. This approach aims to position AI as a tool that enhances, rather than replaces, the rigorous process of writing and validating mathematical proofs.
