TLDR: Dynamic Passage Selector (DPS) is a new method for Retrieval-Augmented Generation (RAG) systems that overcomes the limitations of fixed-size passage selection. Instead of picking a set number of top passages, DPS intelligently learns to dynamically select the most relevant and minimal set of passages for a given query, significantly improving accuracy and reasoning capabilities, especially for complex questions.
Retrieval-Augmented Generation (RAG) systems have become a cornerstone in natural language processing, enhancing the factual accuracy of large language models by retrieving relevant information. However, a common bottleneck in these systems lies within their reranking modules, which traditionally select a fixed number of top passages, often struggling with complex queries that require synthesizing information from multiple sources.
This fixed ‘Top-K’ approach presents a dilemma: choosing a small K might omit crucial information, while a large K can introduce noise and irrelevant content, ultimately degrading the quality of the generated answer. This challenge is particularly evident in multi-hop questions, where the answer is distributed across several documents, and the optimal number of passages needed can vary significantly from one query to another.
Introducing the Dynamic Passage Selector (DPS)
To address this fundamental limitation, researchers have introduced the Dynamic Passage Selector (DPS), a novel reranking framework that redefines passage selection as a supervised learning problem. Unlike conventional methods that score passages independently, DPS is fine-tuned to understand the relationships and dependencies between passages, allowing it to dynamically select the most relevant set of information for generating an answer.
The beauty of DPS lies in its seamless integration; it functions as a ‘plug-and-play’ module, requiring no modifications to existing RAG pipelines. This means it can be easily adopted to enhance current systems without extensive overhauls.
How DPS Works
At its core, DPS transforms the reranking task into a sequence prediction problem. Given a user query and an initial set of retrieved passages, DPS learns to predict an ordered sequence of indices corresponding to the most informative passages. This autoregressive approach allows the model to consider previously selected passages when deciding on the next, capturing crucial inter-passage dependencies and avoiding redundancy. The size of the selected subset is not fixed but is determined dynamically based on the query’s complexity.
The training of DPS involves supervised fine-tuning using a diverse dataset that includes both single-hop and multi-hop question-answering examples, as well as synthetically generated complex queries. This comprehensive training enables DPS to adapt to varying query complexities and identify the minimal yet sufficient set of passages required for accurate answer generation.
Performance and Impact
Extensive evaluations across five challenging benchmarks, including multi-hop datasets like HotpotQA and MuSiQue, and domain-specific datasets such as Legal and CS, demonstrate that DPS consistently outperforms state-of-the-art rerankers and fine-tuning methods. For instance, on the challenging MuSiQue dataset, DPS showed a remarkable F1-score improvement of 30.06% over strong baselines like Qwen3-reranker and 15.4% over RankingGPT.
These results highlight DPS’s ability to dynamically identify contextually relevant passages, moving beyond the limitations of fixed Top-K retrieval. By more effectively aligning retrieval with generation, DPS significantly enhances reasoning capabilities in complex RAG scenarios and shows strong generalization to unseen tasks and domains, even without domain-specific fine-tuning.
Also Read:
- ReasonRank: Advancing Passage Ranking with Enhanced Reasoning
- Keeping Large Language Models Current: A New Framework for Real-Time Knowledge Integration
Scalability and Future Directions
A notable advantage of DPS is its adaptability with various backbone large language models. It consistently improves performance regardless of the underlying generation model, making it a versatile tool for enhancing RAG systems across different scales. Even when paired with smaller models, DPS delivers competitive results, showcasing its efficiency and practicality.
While DPS marks a significant step towards more adaptive and reasoning-aware retrieval, the researchers acknowledge certain limitations, such as constraints imposed by the LLM’s maximum input token length and its reliance on the initial retrieval step. Future work aims to integrate explicit reasoning techniques, explore lightweight versions for zero-shot generalization, and potentially unify retrieval and selection under a single training objective. For more details, you can refer to the full research paper: From Ranking to Selection: A Simple but Efficient Dynamic Passage Selector for Retrieval Augmented Generation.
