TLDR: A study at the University of Maryland developed an AI chatbot for engineering advising, aiming to overcome limitations of traditional advising and mitigate algorithmic bias. By using careful prompt engineering, data curation, and bias detection, the chatbot achieved high scores in accuracy, relevance, and personalization (averaging over 9.5) with no detectable stereotypical biases. Adjusting the AI model’s ‘temperature’ setting significantly improved consistency and reliability, demonstrating that ethical AI can provide effective and equitable academic support.
Choosing a college major is a pivotal decision for many new university students, often fraught with uncertainty. Traditional academic advising, while essential, frequently faces challenges such as lengthy wait times, formal environments that can feel intimidating, and a lack of personalized guidance. These limitations can create significant barriers for students seeking crucial academic direction.
In response to these challenges, Artificial Intelligence (AI) chatbots have emerged as promising tools to enhance accessibility and efficiency in academic advising. These systems offer continuous availability, can reduce social barriers, and deliver standardized information. However, the integration of AI also introduces a critical concern: the potential for algorithmic bias. AI systems can inadvertently generate responses that reflect prejudices related to race, gender, socioeconomic status, or disability, risking the alienation of potential students and undermining the reliability of the AI itself.
A Bias-Aware AI Chatbot for Engineering Advising
A recent study from the University of Maryland’s A. James Clark School of Engineering set out to address this very issue. Researchers developed a specialized AI chatbot designed to assist students with engineering advising, with a core focus on analyzing and mitigating potential biases in its responses. The goal was to create a system that not only provides high-quality academic support but also upholds ethical standards by ensuring fairness and inclusivity.
The research team meticulously developed and tested the chatbot, evaluating its responses based on accuracy, relevance, personalization, and the absence of bias. Their methodology involved a multi-faceted approach to ensure the chatbot was both effective and equitable.
Building an Ethical Advisor
The development process began with extensive data collection. Using web scraping tools like Beautiful Soup and Scrapy, the team gathered comprehensive information about UMD’s engineering programs, including descriptions, course requirements, prerequisites, career pathways, and faculty details. This data was then structured and stored in a JSON format, making it easily retrievable by the chatbot.
For the core of their chatbot, the researchers selected Mistral Medium AI, an open-source Large Language Model (LLM), due to its strong performance in educational applications and its transparency regarding bias mitigation. A crucial aspect of the development was ‘prompt engineering’ – crafting specific instructions and context for the chatbot. This included a base prompt that directed the AI to act as an academic advisor and explicitly avoid stereotypical phrases like “usually” or “typically.” A basic Retrieval Augmented Generation (RAG) system was also implemented, allowing the chatbot to pull relevant information directly from the curated JSON database and append it to its responses.
To detect bias, the team employed regular expressions to identify stereotypical language patterns, such as phrases that link demographic groups with qualifiers like “typically” or “usually,” or comparative terms like “better” or “worse.” This automated detection was complemented by a manual qualitative analysis of the chatbot’s outputs.
Key Findings: Consistency and Zero Bias
The study involved a manual review of 75 chatbot responses, scored on a scale of 1 to 10 for accuracy, relevance, and personalization. Initially, with the AI model’s ‘temperature’ setting at 0.7 (which allows for more varied and creative responses), the chatbot performed well, achieving overall mean scores of 9.09 for accuracy, 8.96 for relevance, and 8.97 for personalization. Importantly, the bias detection process found a 0% bias presence rate at this stage.
However, the researchers observed that some low-scoring responses contained factual inaccuracies. For instance, in one instance, the chatbot incorrectly stated that UMD did not offer a biomedical engineering major.
To enhance consistency and reliability, the team adjusted the AI model’s temperature setting to 0.0, making its responses more deterministic. This change led to significant improvements across all metrics. The mean accuracy score rose to 9.76, relevance to 9.56, and personalization to 9.60. Furthermore, the standard deviations for these scores decreased substantially, indicating much more consistent performance across different types of student queries. Crucially, the bias presence rate remained at 0.0%.
This finding highlights that for critical applications like academic advising, where consistency and reliability are paramount, engineering the model for more deterministic responses is highly beneficial. The proactive bias mitigation strategies, including careful prompt engineering and the explicit instruction to avoid stereotypical language, proved highly effective in preventing biased outputs.
Also Read:
- AI’s Role in Shaping Computational Thinking Education for University Students
- Navigating Mental Health AI: A Framework for Safer Disclosure and Enhanced User Understanding
Looking Ahead
While the study demonstrated remarkable success in developing a high-performing, bias-aware AI chatbot, the researchers acknowledge certain limitations, such as the relatively small sample size and the specificity of the regex-based bias detection. Future work will focus on enhancing user engagement and incorporating more advanced techniques like Retrieval-Augmented Generation with Vector (Semantic) Search to further improve the depth and contextual accuracy of responses.
This research provides valuable insights for building ethical AI systems in higher education, offering tools that can complement traditional advising and address inequities faced by many underrepresented and first-generation college students. For more details, you can read the full research paper here.
