TLDR: This research paper details Oak National Academy’s strategy for building safety guardrails into Aila, their AI-powered lesson planning assistant for UK teachers. It outlines four key guardrails: prompt engineering for curriculum alignment, input threat detection to prevent misuse, an independent AI content moderation agent (IACMA) to classify outputs as safe, content guidance, or toxic, and a human-in-the-loop approach emphasizing teacher review. The paper also discusses evaluation methods, challenges encountered, and the importance of open-source sharing to advance AI safety in education.
The rapid integration of generative AI tools into the education sector has brought about exciting opportunities, but also significant concerns regarding the safety and age-appropriateness of AI-generated content for classrooms. A recent paper by researchers from Oak National Academy, UK, delves into their comprehensive approach to addressing these challenges through the development of Aila, the UK Government’s first publicly available generative AI tool designed to assist teachers with lesson planning.
Aila is specifically built to support teachers in creating national curriculum-aligned lessons suitable for pupils aged 5-16 years. Recognizing the unique sensitivities of this age group, Oak National Academy has implemented a robust framework of four key safety guardrails to mitigate risks associated with AI-generated content.
Four Pillars of Safety in AI Education
The first guardrail is Prompt Engineering. This involves crafting an extensive prompt that codifies Oak’s best practices in pedagogy and cognitive science. It explicitly outlines what Aila should and should not create, ensuring content aligns with the national curriculum and is appropriate for the specified age group and subject. For instance, a lesson on cannabis for older students (11-16 years) would be steered to cover effects, legal, and societal impacts, adhering to the Relationships, Sex and Health Education (RSHE) curriculum.
Next is Input Threat Detection. This layer actively monitors user inputs for anything inappropriate, manipulative, or malicious, particularly prompt attacks aimed at ‘jailbreaking’ the AI. Any flagged input is prevented from reaching the large language model, and repeat offenders are blocked, safeguarding the system from misuse.
The third crucial component is the Independent Asynchronous Content Moderation Agent (IACMA). This AI agent independently assesses Aila’s outputs without context from the user input or Aila’s instructions. It categorizes content into three main types: ‘Safe’ (appropriate for classrooms), ‘Content Guidance’ (appropriate but requires teacher sensitivity, covering topics like upsetting content, discriminatory language, or violence), and ‘Toxic’ (inappropriate and harmful, such as content encouraging self-harm or illegal activities). The IACMA uses a 5-point Likert scale to evaluate severity, with lower scores in ‘content guidance’ triggering warnings and any ‘toxic’ score leading to content blockage and user session termination. The system is intentionally designed to be oversensitive, prioritizing safety even if it means occasional false positives.
Finally, the Human-in-the-loop approach places the teacher as the ultimate safety net. Aila is designed to be a co-planning assistant, with teachers leading the process and reviewing all AI-suggested content before it is used in the classroom. This ensures that the generated material aligns with school policies and is appropriate for their specific pupils.
Also Read:
- Developing Curriculum-Aligned Math Assessments Using Generative AI in Malaysia
- Navigating Trust and Regulation: A Global and GCC Perspective on AI in Education
Evaluating Effectiveness and Future Directions
Oak National Academy undertook rigorous pre-launch evaluations, including testing the IACMA against a corpus of human-created lessons and participating in red-teaming exercises. Post-launch, they continue to evaluate the guardrails using both an illustrative dataset of over 1000 lessons and real user-created lessons (over 45,000). This ongoing review has highlighted challenges, such as the difference between moderating a complete lesson versus moderating it in chunks as it’s generated, and the impact of user input on content moderation.
The paper emphasizes that AI in education is rapidly evolving, and safety approaches must adapt accordingly. As Aila potentially incorporates multimodal elements like imagery and audio, its safety processes will need to expand. A significant contribution of this work is the commitment to cross-sector collaboration: Oak National Academy has made its code, including the prompt, open source, and its content available under an open license. They also plan to share their illustrative dataset, enabling other organizations to build and test their own AI safety guardrails more effectively. This proactive sharing aims to give other educational AI developers a head start in ensuring safety is paramount.
The full research paper provides a deeper dive into these methodologies and findings, and can be accessed here: Building Effective Safety Guardrails in AI Education Tools.
