TLDR: REBot is an AI-powered chatbot designed for academic regulation advising, particularly at Can Tho University. It uses a novel hybrid framework called CatRAG, which combines Retrieval-Augmented Generation (RAG) with GraphRAG. CatRAG employs a category-labeled knowledge graph and an intent classifier to provide accurate, context-aware, and explainable answers to student queries about academic policies. The system achieved a high F1-score of 98.89% in evaluations and has been implemented as a web application for real-time use.
Navigating the complex world of academic regulations can be a daunting task for university students. From understanding scholarship requirements to deciphering study durations, students often spend valuable time sifting through numerous documents or waiting for departmental assistance. This challenge is particularly prevalent in institutions like Can Tho University (CTU) in Vietnam, where information is spread across various official sources, leading to confusion and inefficiency.
Introducing REBot: An AI-Powered Academic Advisor
To address this persistent issue, researchers have developed REBot, an innovative AI-enhanced advisory chatbot. REBot is designed to provide CTU students with fast, accurate, and up-to-date answers regarding academic regulations. At its core, REBot leverages a sophisticated hybrid retrieval–reasoning framework known as CatRAG.
CatRAG: The Brain Behind REBot
CatRAG stands for Category-guided Retrieval-Augmented Generation. It represents a significant advancement by integrating two powerful AI techniques: traditional Retrieval-Augmented Generation (RAG) and GraphRAG. While RAG is excellent for grounding responses in factual evidence, it can sometimes struggle with ambiguous queries. GraphRAG, on the other hand, excels at capturing structured policy relations through knowledge graphs but can lack generative flexibility when used in isolation. CatRAG intelligently combines these strengths, unifying dense retrieval from a vector store with symbolic reasoning over a regulation-specific knowledge graph.
The ‘Cat’ in CatRAG highlights its category-guided design. This means that when a student asks a question, a lightweight intent classifier first routes the query to the most relevant academic domain, such as enrollment, graduation, or assessment. This categorization helps narrow down the search space, ensuring that the chatbot focuses on the most pertinent information.
How CatRAG Works
CatRAG operates through two main algorithms: CatRAG-Construct and CatRAG-Query.
-
CatRAG-Construct: This algorithm is responsible for building REBot’s comprehensive knowledge base. It processes academic documents, extracting text, normalizing it, and enriching its semantics with a custom dictionary of regulation terms. The text is then broken down into smaller ‘chunks,’ which are indexed in a vector store for quick semantic retrieval. Simultaneously, these chunks are linked to entities and organized into a hierarchical, category-labeled knowledge graph. This graph captures complex relationships between different pieces of information, allowing for explainable, graph-structured evidence.
-
CatRAG-Query: When a student poses a question, this algorithm springs into action. The query is first embedded and classified into an academic category. CatRAG then simultaneously retrieves relevant information from both the global vector store (using RAG) and the corresponding knowledge subgraph (using GraphRAG). Entities and their relationships are expanded to enrich the context, which is then passed to a large language model (LLM) like gpt-4o-mini or Mistral for generating the final, coherent answer. This hybrid approach ensures both efficiency in recalling information and precision in understanding domain-specific context.
REBot’s Architecture and Components
The REBot framework is built upon five interconnected components:
-
Knowledge Extraction: Processes PDFs, cleans text, segments it, and enriches it using Named Entity Recognition (NER) and relation extraction, with embeddings from PhoBERTv2 for Vietnamese language representation.
-
Student Affairs Classification: Categorizes student queries into five regulation domains using fastText, chosen for its efficiency and accuracy with short Vietnamese texts.
-
REBot Knowledge Graph: A Neo4j database that organizes knowledge into domain-specific graphs with three layers: categories, contexts, and entities.
-
Vectorization and Embeddings: Uses LlamaIndex to build a vector index for RAG, combined with PhoBERTv2 embeddings, and integrates GraphRAG to unify vector retrieval with graph structures.
-
Response Refinement: Employs LLMs like Mistral and ChatGPT to refine retrieved content into fluent, context-aware answers.
Also Read:
- KG-R1: A Unified Agent for Efficient and Adaptable Knowledge Graph Reasoning
- Improving Dialogue Flow in Persian Chatbots with a Hybrid AI Model
Impressive Performance and Real-World Application
REBot has been rigorously tested on a curated dataset of 1,319 question-answer pairs related to academic regulations. The results are highly promising, with CatRAG combined with gpt-4o-mini achieving a state-of-the-art F1-score of 98.89% at a threshold of 0.6. While CatRAG does incur a slightly longer response time (around 7 seconds per query compared to 5 seconds for standard RAG) due to its graph-based retrieval and node similarity computations, this latency remains acceptable for real-time use, given the enhanced accuracy and contextual depth it provides.
The practical value of REBot is demonstrated through its implementation as a web application. This user-friendly interface allows students to interact with the chatbot in real-time, receiving clear and in-depth explanations to their queries, such as the number of semesters in an academic year or foreign language graduation requirements. This system offers a significant improvement over traditional methods, laying the groundwork for more flexible and domain-focused AI helpers in education.
The research paper detailing REBot and CatRAG can be found here: REBot: From RAG to CatRAG with Semantic Enrichment and Graph Routing.
