TLDR: VideoAgent is a novel multi-agent framework designed to automatically generate personalized scientific videos from research papers. It processes a paper into a fine-grained asset library, then, guided by user requirements, orchestrates a narrative flow that synthesizes both static slides and dynamic animations. The framework ensures multimodal content synchronization and is evaluated by SciVidEval, a comprehensive benchmark. Experiments demonstrate that VideoAgent significantly outperforms commercial services and achieves near human-level quality in scientific communication and knowledge transfer.
Scientific research is vital, but effectively sharing complex findings can be a challenge. Traditional methods like static posters and slides often fall short in engaging a wider audience and illustrating dynamic processes. Imagine transforming a dense research paper, filled with specialized terms, data charts, and intricate logic, into a compelling video that simplifies understanding and boosts engagement.
This is precisely the problem that a new framework called VideoAgent aims to solve. Developed by Xiao Liang, Bangxin Li, Zixuan Chen, Hanyue Zheng, Zhi Ma, Di Wang, Cong Tian, and Quan Wang from the School of Computer Science and Technology at Xidian University, VideoAgent is a novel multi-agent system designed for the personalized synthesis of scientific videos. You can find the full research paper here: VIDEOAGENT: PERSONALIZED SYNTHESIS OF SCIENTIFIC VIDEOS.
VideoAgent tackles two core challenges in scientific video generation: personalized and dynamic orchestration, and multimodal content synchronization. Existing tools often rely on fixed templates, lacking the ability to adapt to user-guided synthesis or decide when to use static slides versus dynamic animations. Furthermore, ensuring that narration aligns perfectly, both temporally and semantically, with visual content is a complex task.
How VideoAgent Works
The framework operates in a multi-stage process, starting with a conversational interface that allows users to specify their requirements. Here’s a breakdown of its key components:
Document Parser: This initial stage takes a source paper PDF and breaks it down into a fine-grained library of multimodal assets. This includes extracting text, figures, tables, and equations. It then summarizes chapters and generates textual descriptions for visual assets, storing everything in a structured JSON file.
Requirement Analyzer: This module acts as a dialogue-guided interface, translating user inputs into a structured JSON configuration. Users can specify functional requirements (like enabling animations or detailed explanations for figures) and technical specifications (such as video duration, resolution, or presentation style). This ensures the generated video is highly personalized to the user’s communication goals.
Personalized Planner: This is the core orchestrator. It iterates through the chapter summaries and selected assets to create a detailed storyboard. It decides on content selection, determines the number of slides needed, and generates code for both static slides (using python-pptx) and dynamic animations (using python-manim). The planner is crucial for deciding whether to use a static slide or a dynamic animation to best convey complex ideas.
Multimodal Synthesizer: In the final stage, the generated slides are converted into high-resolution images. A Text-to-Speech agent creates narration audio and synchronized subtitles. The duration of the audio dictates the display time for static slides or adjusts the speed of animations. Finally, MoviePy is used to combine all elements—slide images, animation clips, narration audio, and subtitles—into a final MP4 video file.
Evaluating Effectiveness with SciVidEval
To rigorously evaluate VideoAgent, the researchers also introduced SciVidEval, a comprehensive benchmark. This suite combines automated metrics for assessing narration quality, visual quality, and audio-visual synchronization. Crucially, it also includes a Video-Quiz-based human evaluation to directly measure knowledge transfer. Graduate students are tasked with answering multiple-choice questions based solely on watching the generated videos, providing a direct measure of how effectively scientific insights are conveyed.
Also Read:
- Charting the Path to Self-Driving Science with AI Agents
- AI Unites Visuals and Policies for Smarter Disaster Damage Assessment
Impressive Results
Extensive experiments show that VideoAgent variants consistently outperform existing commercial scientific video generation services. For instance, the Gemini-2.5 Pro variant achieved high scores across visual quality and synchronization, demonstrating its superior ability to create a cohesive and accurate audio-visual narrative. In terms of knowledge transfer, the Gemini-2.5 Pro variant achieved an impressive 87.5% accuracy in human evaluations, closely matching author-created videos and significantly outperforming commercial services. This confirms VideoAgent’s effectiveness in conveying complex scientific concepts to a human audience, approaching human-level quality in scientific communication.
VideoAgent represents a significant step forward in automating the generation of scientific videos, making knowledge dissemination more effective, engaging, and personalized.
