TLDR: WebGen-V is a new benchmark and framework for instruction-to-HTML generation that improves web design quality and evaluation. It achieves this by using an agentic crawling system to collect real-world webpages and then representing them in a structured, section-wise format with localized UI screenshots and metadata. This enables a fine-grained, multimodal evaluation protocol that provides precise feedback for iterative refinement, leading to more accurate and visually faithful AI-generated web pages compared to traditional full-page methods.
The field of web page generation, where large language models (LLMs) create HTML from instructions, is rapidly growing. However, existing methods often struggle with the complexity and visual richness of real-world websites. Traditional benchmarks typically rely on full-page screenshots and lengthy raw HTML code, which can limit the quality of generated designs and make detailed evaluation difficult.
A new research paper introduces WebGen-V, a novel benchmark and framework designed to enhance both the data quality and the granularity of evaluation for instruction-to-HTML generation. This framework addresses the shortcomings of previous approaches by introducing a more structured and detailed way to represent web pages.
Key Innovations of WebGen-V
WebGen-V brings three significant advancements to the table:
First, it features an unbounded and extensible agentic crawling framework. This system continuously collects real-world web pages, allowing for a diverse and ever-growing dataset that can also augment existing benchmarks. This ensures that the models are trained and evaluated on data that truly reflects the variety of modern web design.
Second, WebGen-V employs a structured, section-wise data representation. Unlike benchmarks that provide only raw HTML and page-level screenshots, WebGen-V breaks down web pages into distinct sections. For each section, it integrates metadata, localized UI screenshots, and JSON-formatted text and image assets. This explicit alignment between content, layout, and visual components enables much more detailed multimodal supervision for LLMs.
Third, the framework introduces a section-level multimodal evaluation protocol. This allows for high-granularity assessment by aligning text, layout, and visuals at the individual section level. This goes beyond whole-page judgments, enabling models to be assessed on localized content understanding and generation quality, which is crucial for identifying subtle design flaws.
How WebGen-V Works
The WebGen-V framework operates through two core modules: a Crawling Module and an Evaluation Module, both supported by a generic Processor. The Crawling Module acquires and preprocesses real-world web pages, transforming them into the structured data format. It uses a keyword-based Seed Finder to discover URLs, then a hybrid renderer (HTTP requests for static, Playwright for dynamic) to capture full HTML, screenshots, and assets. The Processor then decomposes these raw pages into sections, identifying functional blocks like hero sections or footers, and extracting cropped screenshots, text, and JSON metadata for each. GPT-5 is used to classify image assets semantically.
The Evaluation Module assesses generated web pages. It renders the model-generated HTML and applies the same Processor to obtain a structured representation. A multimodal LLM (GPT-5) then evaluates each section for text correctness, visual alignment, readability, and multimodal coherence. Crucially, this evaluation doesn’t compare against a single “correct” reference layout, but rather assesses whether the design logically fulfills the described intent based on the instruction. The results are aggregated into structured feedback, including quantitative scores and qualitative rationales.
This feedback can then be used in a Generation–Evaluation–Refinement process. If a section’s score falls below a certain threshold, the feedback is reintroduced to the model, prompting a targeted regeneration of that specific section. This iterative refinement allows models to address localized issues without disrupting the overall design flexibility.
Also Read:
- JudgeSQL: Enhancing Text-to-SQL Accuracy Through Intelligent Selection
- Upgrading Multimodal AI Data: The VERITAS Pipeline
Experimental Validation and Impact
Experiments with state-of-the-art LLMs like GPT-5, Gemini-2.5-Pro, and Claude-Opus-4.1 validate the effectiveness of WebGen-V. The structured, section-wise representation consistently enhances evaluation quality and leads to improved generation quality. It more accurately detects human-injected degradations compared to traditional full-page evaluations, demonstrating its superior ability to localize and identify realistic layout, text, and media defects. Ablation studies confirm that fine-grained, section-wise cues (structured text and section-wise screenshots) are essential for strong performance.
Furthermore, the framework’s processor can adapt existing HTML benchmarks into this structured format, providing an alternative data source beyond real-world crawling. This means older datasets can be recontextualized for modern instruction-to-HTML generation tasks.
WebGen-V addresses the “resolution barrier” for long webpages by capturing section-wise screenshots, preserving full visual fidelity. It also moves towards fine-grained multimodal understanding, mitigating the compression issues often seen with full-page visual inputs in LLMs. The research also notes that structured feedback can help more economical models achieve quality comparable to premium ones after refinement, suggesting a favorable cost-performance trade-off.
In conclusion, WebGen-V offers a unified pipeline for instruction-to-HTML generation, from data acquisition to structured multimodal assessment. By focusing on high-granularity, section-level analysis, it paves the way for more realistic, visually faithful, and precisely evaluated AI-powered web design. You can read the full paper for more details here.
