TLDR: Hermes 4 is a new family of AI models that combines advanced, multi-step reasoning with a wide ability to follow instructions. This report details its innovative data creation, training methods, and extensive evaluation across various benchmarks like math, coding, and general knowledge. The models are designed for transparency and reproducibility, with their weights publicly released.
NousResearch has introduced Hermes 4, a new family of hybrid reasoning models designed to integrate structured, multi-step reasoning with a broad capacity for instruction following. This technical report details the comprehensive approach taken in developing these models, from data curation and synthesis to training and evaluation, emphasizing transparency and reproducibility.
The development of Hermes 4 addressed significant challenges in creating a model that excels in both complex reasoning tasks and general instruction adherence. The team outlined innovative solutions employed to scale these capabilities effectively. The models underwent rigorous evaluation across a wide array of benchmarks, including mathematical reasoning, coding, knowledge, comprehension, and alignment, providing both quantitative performance metrics and qualitative behavioral analysis.
Data Strategy and Synthesis
A cornerstone of Hermes 4 is its unique data synthesis and curation strategy. The dataset comprises approximately 5 million samples and 19 billion tokens, meticulously designed to instill advanced reasoning while maintaining a strong foundation in general knowledge. This hybrid dataset includes 3.5 million reasoning-focused samples, which are notably token-heavy to accommodate extensive ‘thinking traces’ up to 16,000 tokens long, alongside 1.6 million non-reasoning instruction examples.
The data generation process leverages a graph-based synthetic data generator called DataForge. Inspired by AgentInstruct, DataForge creates conversational data through a series of transformations and instruction generations. It processes pre-training seed data, transforming passages into various document types, generating instructions (contextual or standalone), and then producing answers using specialized LLMs. A crucial step involves an LLM judge, with different weights from the answer model, reviewing and iterating on instruction-answer pairs to ensure quality.
Further enhancing the dataset, Hermes 4 utilizes rejection sampling with Atropos, an open-source reinforcement learning environment manager. This process generates a large corpus of verified reasoning trajectories across various environments. These environments include tasks for precise answer formatting, complex instruction following, methodical problem-solving (from Internbootcamp), schema adherence for structured outputs like JSON, and tool use, where the model learns to integrate external tools like Python interpreters into its reasoning process.
To ensure comprehensive coverage of target domains, the team employed two techniques: taxonomies for data-scarce areas, using LLMs to enumerate subdomains and prompts, and PersonaHub, which simulates human users via synthetic personas to generate diverse application and script implementation tasks.
Training Methodology
Hermes 4 was trained using a modified version of TorchTitan, building upon existing models like Llama 3.1 (405B and 70B versions) and Qwen3 14B. Given the highly varied lengths of samples in the dataset, an efficient packing strategy called First-Fit Decreasing was used, achieving over 99.9% batch efficiency. Flex Attention was implemented to ensure that attention mechanisms were confined to individual samples within a packed batch, and only tokens generated by the ‘assistant’ role contributed to the training loss.
The training was conducted on 192 NVIDIA B200 GPUs, utilizing a mix of Distributed Data Parallelism, Tensor Parallelism, and Fully Sharded Data Parallelism. A standard cosine learning rate schedule was followed over 9,000 steps, with a global batch size of 384 samples and a context length of 16,384 tokens.
A notable aspect of the training was the introduction of a second supervised fine-tuning stage specifically to control reasoning length. The 14B model, in particular, frequently reached its maximum context length during reasoning tasks. To address this, synthetic reasoning traces were generated, and a special `` token was inserted at 30,000 tokens. By loss-masking, only this `` token (and the end-of-sequence token) received gradient updates, teaching the model to terminate its reasoning at a specific point without altering its core reasoning distribution, thereby mitigating risks of model collapse often seen with recursive self-generated data training.
Also Read:
- Boosting Language Model Math Abilities Through Data Analysis
- Integrating Knowledge Graphs for Advanced Multi-hop Question Answering in Language Models
Evaluation and Behavioral Insights
The evaluation of Hermes 4 was extensive, comparing it against a selection of recent frontier open-weight models. All generated samples were logged and publicly released to ensure full transparency. The evaluation harness was designed to use an OpenAI-compatible chat completions endpoint, promoting reproducibility across different inference engines and hardware configurations.
Benchmarks included Lighteval for math and multiple-choice tasks, the EQBench collection for subjective evaluations, and Atropos for LiveCodeBench and custom assessments. Atropos, functioning as an evaluation framework, offered single-file evaluations, detailed sample-level logging, and performance-conscious execution that overlapped inference and scoring to maximize efficiency.
A unique internal benchmark, RefusalBench, was developed to measure how often the model refused prompts across 32 categories, including those related to safety. Hermes 4 demonstrated strong performance in this area, showing fewer general refusals while appropriately handling sensitive topics with inverted rewards.
Beyond quantitative scores, Hermes 4 exhibited distinctive qualitative behaviors. Under standard prompting, it showed greater ‘contextual fidelity’ compared to other models, which often displayed ‘policy rigidity’ by foregrounding AI identity disclaimers. Hermes 4 was more adept at interpreting fictional prompts as role-play and generating in-character responses. It also demonstrated superior ‘stylistic transferability’ in creative writing tasks, approximating authorial rhythm and diction rather than just surface-level references or paraphrases.
Furthermore, Hermes 4 proved highly sensitive to prompt engineering. Customizing system prompts for anti-sycophancy led to a deeper shift in its chain-of-thought, actively steering interactions away from deference. Modifying chat templates, such as changing the ‘assistant’ role to ‘me’, resulted in the model adopting a first-person, peer-like persona with fewer meta-disclaimers, indicating a high degree of behavioral plasticity.
The Hermes 4 models, with their weights publicly available at HuggingFace, represent a significant step forward in developing open-weight reasoning models that are both powerful and transparent, offering capabilities comparable to frontier systems while fostering open research and development.
