TLDR: ReasonRank is a new passage reranker that uses an automated data synthesis framework to create reasoning-intensive training data and a two-stage training approach (supervised fine-tuning followed by reinforcement learning with a multi-view reward). It significantly outperforms existing models on complex ranking tasks, is more efficient than pointwise rerankers, and achieves state-of-the-art results on benchmarks like BRIGHT.
In the evolving landscape of information retrieval, large language models (LLMs) have shown remarkable promise in ranking passages to improve search results. However, a significant challenge remains: equipping these models with strong reasoning abilities, especially for complex queries that go beyond simple keyword matching. Traditional training data often falls short, leading to performance gaps in real-world, reasoning-intensive scenarios.
A new research paper, “ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability,” introduces an innovative solution to this problem. Authored by Wenhan Liu, Xinyu Ma, Weiwei Sun, Yutao Zhu, Yuchen Li, Dawei Yin, and Zhicheng Dou, the paper details a novel approach to train rerankers that can handle intricate reasoning tasks with high efficiency.
Addressing the Data Gap
The core issue identified by the researchers is the scarcity of training data that truly demands complex reasoning. To overcome this, they developed an automated framework for synthesizing high-quality, reasoning-intensive training data. This framework gathers queries and passages from diverse domains, including complex question-answering platforms like StackExchange, coding challenges from Leetcode, mathematical problems from the MATH dataset, and even traditional web search queries from MSMARCO. A powerful reasoning model, DeepSeek-R1, is then employed to generate accurate training labels, including detailed reasoning steps and the correct ranking of passages. To ensure the reliability of this synthesized data, a self-consistency filtering mechanism is applied, discarding any low-quality samples.
A Two-Stage Training Breakthrough
With this rich, synthesized dataset, the researchers propose a two-stage training approach for their ReasonRank model. The first stage, called “cold-start supervised fine-tuning (SFT),” focuses on teaching the LLM (specifically, Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct) to understand and generate reasoning patterns for listwise ranking. This initial phase helps the model grasp the fundamental logic required for complex ranking tasks.
The second stage leverages reinforcement learning (RL) to further refine the model’s ranking capabilities. Unlike previous methods that might rely on a single metric like NDCG@10, ReasonRank introduces a “multi-view ranking reward.” This innovative reward system considers not only the traditional ranking metric but also “Recall@10” (to ensure relevant passages are not overlooked) and “Rank-biased overlap (RBO),” which measures the similarity between the model’s output and the ideal ranking. This multi-faceted reward helps the model explore better reasoning strategies and improve its overall ranking performance, especially in the context of sliding-window listwise ranking.
Impressive Performance and Efficiency
Extensive experiments on challenging reasoning-intensive benchmarks like BRIGHT and R2MED demonstrate ReasonRank’s superior performance. The model significantly outperforms existing baselines, even with smaller model sizes. For instance, ReasonRank (7B) often surpasses 32B-scale baselines. Furthermore, ReasonRank exhibits remarkable efficiency. Despite its reasoning capabilities, the listwise ReasonRank is 2 to 2.7 times faster than pointwise rerankers like Rank1. This efficiency stems from ReasonRank processing multiple passages with a single reasoning chain, drastically reducing the number of output tokens needed.
The paper also highlights ReasonRank’s strong generalization ability, showing competitive performance on traditional information retrieval benchmarks like BEIR. Further enhancements, such as using higher-quality initial retrieval results and optimizing sliding window parameters, pushed ReasonRank to achieve state-of-the-art performance on the BRIGHT leaderboard.
Also Read:
- Unlocking Deeper Search: DIVER’s Approach to Reasoning-Intensive Information Retrieval
- Bridging Knowledge and Logic in Language Models with UR2
Looking Ahead
While ReasonRank marks a significant advancement, the authors acknowledge areas for future improvement. They plan to incorporate non-reasoning data into training to allow the model to seamlessly adapt to varying query difficulties. Exploring other LLM backbones beyond the Qwen2.5 series is also a future direction. Additionally, the current reliance on a sliding window strategy could be replaced by full-list ranking approaches, which have shown promise in handling even larger sets of passages in a single pass. You can find the full research paper here.
