TLDR: DOCREWARD is a new AI model that evaluates documents based on their visual structure and style, not just text quality. Trained on a large dataset of paired documents with identical content but varying visual professionalism, it significantly outperforms GPT-4o and GPT-5 in accurately assessing human preferences for document design. It also proves effective in guiding AI agents to generate more visually professional and human-preferred documents.
Recent advancements in artificial intelligence have brought about sophisticated agentic workflows, capable of automating complex tasks such as generating professional documents. While these systems have excelled in producing high-quality text, a significant challenge has remained: the neglect of visual structure and style. These elements are crucial for making documents readable, engaging, and truly professional.
The core issue stems from the absence of suitable “reward models” that can guide AI agents to prioritize and improve the structural and stylistic quality of documents. Current models primarily focus on the textual content itself, overlooking how a document looks and feels.
To address this critical gap, researchers have introduced DOCREWARD, a novel Document Reward Model specifically designed to evaluate documents based on their structure and style. This innovative model aims to bring a new level of professionalism to AI-generated documents by focusing on their visual presentation.
How DOCREWARD Was Built
The development of DOCREWARD involved creating a unique and extensive dataset called DOCPAIR. This multi-domain dataset comprises 117,000 paired documents, spanning 32 different domains and 267 document types. Each pair consists of a document with high professionalism in structure and style, and a counterpart with low professionalism. Crucially, both documents in a pair share identical textual content. This design ensures that DOCREWARD learns to assess visual attributes independently of the text’s inherent quality.
The construction of DOCPAIR involved curating high-quality professional documents, expanding these source documents using various generation agents (like GPT-4o, Claude Sonnet 4, and GPT-5), and then meticulously ranking them. Human-authored documents were always preferred over agent-generated ones, and GPT-5 was used as a reliable tool to rank synthetic documents against a human-authored reference.
DOCREWARD itself is trained using a technique called Bradley-Terry loss. This method optimizes the model to assign higher scores to preferred documents and penalizes predictions that contradict the annotated rankings, effectively teaching it human preferences for document aesthetics.
Defining Professional Structure and Style
The paper clearly outlines what DOCREWARD considers professional structure and style:
- Structure: This includes proper use of white space, appropriate margins, clear section breaks, well-structured text alignment, adequate paragraph spacing, proper indentation, inclusion of page headers and footers, and logical, coherent organization of content.
- Style: This encompasses appropriate font choices (type, size, color, readability), clear heading styles, effective use of emphasis (bold, italics), bullet points, numbering, and consistent formatting.
By focusing on these specific elements, DOCREWARD can comprehensively evaluate document professionalism in a way that is agnostic to the textual content.
Also Read:
- AI Models Evaluate User Interfaces: A New Benchmark for Design Feedback
- Enhancing Text-to-Image Creation with Collaborative AI Agents
Impressive Performance and Real-World Utility
DOCREWARD’s effectiveness was rigorously tested through both intrinsic and extrinsic evaluations.
In intrinsic evaluations, DOCREWARD-7B achieved an impressive 89.22% human preference accuracy, significantly outperforming leading models like GPT-4o and GPT-5 by 30.6 and 19.4 percentage points, respectively. This demonstrates its superior ability to align with human judgments regarding document structure and style.
For extrinsic evaluation, DOCREWARD was used as a reward model to guide document generation agents. It achieved a significantly higher win rate of 60.8% in human evaluations, compared to GPT-5’s 37.7%. This proves its practical utility in helping AI agents produce documents that humans prefer, without altering the underlying generation agent itself.
Interestingly, analysis of DOCREWARD’s attention maps revealed that the model focuses on visual and formatting cues, such as headings, numbering, page headers and footers, bullet points, table borders, and even page corners to implicitly check for uniform margins and balanced whitespace. This confirms its specialization in visual document quality rather than just text.
In conclusion, DOCREWARD represents a significant step forward in document AI, offering a powerful tool to enhance the visual professionalism of AI-generated content. For more details, you can read the full research paper here.
