Advancing Conversational Search with Intelligent Query Reformulation and Result Merging

TLDR: This research paper details the CFDA & CLIP team’s participation in the 2025 TREC Interactive Knowledge Assistance Track (iKAT). The team focused on enhancing personalized conversational search through advanced query reformulation (rewriting user queries for clarity) and retrieval fusion (combining search results from multiple rewritten queries). Their methodology involved a unified pipeline incorporating LLM-based query rewriting, sparse retrieval, neural reranking, and LLM-driven response generation. Key findings demonstrate that these strategies, particularly when combined with reranking and fusion applied before reranking, significantly improve the robustness and effectiveness of conversational search systems in both interactive and offline evaluation settings.

In the evolving landscape of artificial intelligence, conversational search stands out as a critical area, moving beyond simple keyword queries to dynamic, multi-turn interactions. Users expect systems to understand their evolving intent, handle ambiguous language, and adapt to context over time. This challenge was at the heart of the 2025 TREC Interactive Knowledge Assistance Track (iKAT), which introduced both interactive, real-time tasks and traditional offline evaluations.

A team from CITI, Academia Sinica, along with collaborators from National Chengchi University, presented their work, “CFDA & CLIP at TREC iKAT 2025: Enhancing Personalized Conversational Search via Query Reformulation and Rank Fusion.” Their research explores key strategies like query rewriting and retrieval fusion to build robust and efficient conversational search pipelines. You can read the full paper here: CFDA & CLIP at TREC iKAT 2025.

The Core Challenge: Conversational Search

Traditional search engines excel at processing single, well-formed queries. However, conversational search demands more. User queries in a dialogue are often incomplete, ambiguous, or rely heavily on previous turns. Systems need to maintain a dialogue state, infer implicit user intent, and continuously adapt. While past research often relied on static offline evaluations, the iKAT 2025 interactive task pushed for systems capable of operating in real-time, facing unpredictable user behavior simulated by a hidden large language model (LLM) API.

Key Strategies: Query Rewriting and Retrieval Fusion

The researchers focused on two main approaches to tackle these challenges:

• Query Rewriting: This involves transforming an ambiguous or incomplete user utterance, along with the dialogue history and a Personal Text Knowledge Base (PTKB), into a clear, self-contained query. This allows existing ad-hoc retrievers to be used effectively. The team explored frameworks like LLM4CS and CHIQ-AD, which leverage large language models to generate contextually coherent rewrites. They also looked into AdaRewriter, which generates multiple candidate rewrites and selects the best one using a reward model.
• Retrieval Fusion: This strategy combines results from multiple rewritten queries or different retrieval methods. The paper highlights Reciprocal Rank Fusion (RRF) as a powerful technique to merge ranked lists of passages, improving the overall robustness and coverage of search results.

The Pipeline: From Query to Response

The team’s unified conversational retrieval pipeline consists of four main stages:

1. Query Rewriting: As described above, this stage reformulates the user’s input into a precise query.
2. Passage Retrieval: Using the rewritten query, an initial set of relevant passages is retrieved from a large collection. The team used SPLADE, a sparse expansion model known for its effectiveness.
3. Passage Reranking: The initial set of passages is then refined by neural rerankers (like DeBERTaV3 and BGE) to improve precision by modeling fine-grained interactions between the query and passages.
4. Response Generation: Finally, an LLM generates a grounded response based on the reranked passages and relevant PTKB statements. For the offline task, this stage also involved PTKB classification and passage summarization.

Experimental Insights and Findings

The team submitted several runs for both interactive and offline tasks, testing different combinations of their strategies. Their experiments on iKAT 2023 and 2024 datasets yielded several key observations:

• Query Rewriting is Essential: Both CHIQ-AD and LLM4CS significantly outperformed a retriever-only baseline, confirming that context-aware query rewriting is crucial for conversational search.
• Reranking Boosts Precision: Adding a neural reranker consistently improved performance across all rewriting strategies, highlighting its vital role in refining retrieval accuracy.
• Fusion Enhances Robustness: RRF-based fusion achieved the strongest overall performance, demonstrating that combining multiple rewrites makes the system more robust to conversational ambiguity. Notably, applying RRF before reranking yielded better results, as it exposed the reranker to a richer and more diverse set of candidates.
• Trade-offs in Models: While advanced prompt-based rewriting (LLM4CS, CHIQ-AD) was effective, fine-tuned models like CHIQ-FT showed limitations due to restricted context windows, struggling with long dialogue histories.
• Optimizing Candidates: For AdaRewriter, increasing the number of candidate rewrites generally improved performance, but beyond a certain point (around 10 candidates), the improvements became marginal compared to the increased computational cost.

Also Read:

• Boosting Information Retrieval with Chunk-Based Knowledge Generation
• Optimizing In-Context Learning: A Kernelized and Information-Theoretic Approach to Example Selection

Looking Ahead

The study underscores the importance of modular pipelines that effectively balance rewriting, reranking, and fusion under interactive constraints. The researchers identified several promising future directions, including optimizing for latency in real-time systems, developing adaptive query strategies that dynamically decide between single-query or multi-query fusion, deeper integration of Personal Text Knowledge Bases, and enhancing passage summarization techniques for response generation.

This work provides valuable insights into building more adaptive and deployable conversational search systems, pushing the boundaries of how AI assists users in complex, multi-turn interactions.