TLDR: QuesGenie is a multimodal question generation system that automatically creates various question types (MCQ, true/false, fill-in-the-blanks, matching) from different content formats like text, diagrams, and audio. It uses components for input handling, question generation (T5 for text, ChartGemma for visuals), and reinforcement learning from human feedback to improve question quality, aiming to ease educators’ workload and provide diverse practice materials for students.
In today’s fast-paced educational landscape, learners have access to a wealth of information, but often struggle to find enough practice materials specifically tailored to these resources. Educators, on the other hand, face the time-consuming and labor-intensive task of creating assessment questions manually. This can lead to inconsistencies and a lack of variety in practice questions.
Addressing these challenges, a new project titled “QuesGenie: Intelligent Multimodal Question Generation” introduces an innovative system designed to automatically generate diverse question types from various content formats. This system aims to significantly reduce the workload for educators while providing students with a powerful tool for exam preparation and deeper understanding.
What is QuesGenie?
QuesGenie is a comprehensive system that can process multiple input formats, including presentation slides, PDF documents, and audio recordings. From these sources, it generates different types of questions such as true/false, multiple-choice questions (MCQ), fill-in-the-blanks, and matching questions. Crucially, it can generate questions from both text and images found within the input materials.
The system is built around four main components: handling multimodal inputs, generating questions, improving through reinforcement learning from human feedback (RLHF), and an interactive user interface. This foundation allows for automated, scalable, and intelligent question generation, balancing efficiency, robust functionality, and a smooth user experience.
How Does It Work?
QuesGenie begins by preprocessing input sources, converting them into manageable “chunks” – either text-based or image-based. Audio recordings are transcribed using Whisper, while PowerPoint and PDF documents yield both text and image data. For visual content, only diagrams are selected for question generation, using a specialized diagram classification model called DiaClass.
For text-based questions, the system leverages the T5 (Text-to-Text Transfer Transformer) model. T5 is fine-tuned on datasets like SQuAD for generating question-answer pairs, RACE for creating plausible distractor options for MCQs, and BoolQ for true/false questions. Fill-in-the-blank questions are generated by identifying key terms using the Python Keyphrase Extraction (PKE) toolkit and replacing them with blanks.
Visual question generation is handled by ChartGemma, a model specifically designed for chart reasoning. After images are classified as diagrams by DiaClass, ChartGemma generates questions and answers based on the visual elements. This extends the system’s capability beyond just text, allowing it to interpret and query information presented in diagrams.
Learning and Improvement
A key aspect of QuesGenie is its ability to learn and improve. It incorporates reinforcement learning, specifically Proximal Policy Optimization (PPO), to align question generation with human preferences. Users can provide feedback through a 5-star rating system, which helps train a reward model (DistillRoBERTa) to evaluate the relevance and quality of generated questions. While human feedback is ideal, the research explored using AI-generated feedback and noted that even a pre-trained reward model could offer slight improvements in question structure and quality.
Also Read:
- SATQuest: A New Approach to Evaluating and Enhancing AI’s Logical Reasoning
- Enhancing Omni-Modal Language Models: A New Framework to Combat Hallucinations
Looking Ahead
The project lays significant groundwork for multimodal question generation. Future enhancements could include collecting more user feedback to further refine the reinforcement learning process, allowing users to upload sample questions or past exam papers to generate similar new questions, and introducing adaptive difficulty levels. Support for multilingual question generation and integration into a full-featured educational platform with student dashboards and performance analytics are also envisioned. This work represents a valuable contribution to the intersection of AI and education, promising far-reaching benefits for learners and educators alike.
You can read the full research paper here: QuesGenie: Intelligent Multimodal Question Generation.
