TLDR: FLAIRR-TS is a novel framework that uses a system of AI agents (Forecaster, Refiner, Retrieval) to iteratively refine natural language prompts for Large Language Models (LLMs) at test-time. This approach significantly improves time series forecasting accuracy by dynamically adapting prompts and integrating relevant historical context, effectively reducing the need for manual prompt engineering and outperforming existing baselines without model fine-tuning.
Large Language Models (LLMs) are increasingly being explored for their potential in various tasks, and time series forecasting (TSF) is no exception. However, effectively leveraging LLMs for TSF often hits a roadblock: the need for meticulously crafted natural language prompts. This process, known as prompt engineering, is typically manual, time-consuming, and requires significant expertise for each new forecasting task.
A new research paper introduces FLAIRR-TS, a novel framework designed to overcome this prompt engineering bottleneck. FLAIRR-TS stands for Forecasting LLM-Agents with Iterative Refinement and Retrieval for Time Series. It’s an innovative test-time prompt optimization system that allows LLMs to generate high-quality forecasts without the need for extensive pre-processing, fine-tuning, or intermediate code generation.
How FLAIRR-TS Works: An Agentic Approach
At its core, FLAIRR-TS operates as a multi-agent system, where different AI agents collaborate to refine the forecasting process. This system includes three main agents:
- The Forecaster Agent: This agent is responsible for generating the initial time series predictions. It uses a prompt that is continuously improved by the Refiner Agent.
- The Refiner Agent: Acting as a meta-optimizer, this agent analyzes the history of previous forecasting attempts and their errors. Based on this analysis, it provides specific, actionable feedback to refine the prompt used by the Forecaster Agent. This iterative refinement continues until the forecast quality meets a predefined criterion or a maximum number of iterations is reached.
- The Retrieval Agent: Inspired by Retrieval Augmented Generation (RAG), this agent enhances the Forecaster Agent’s input by finding and providing relevant historical time series segments. These segments act as illustrative examples of past patterns, grounding the forecasts in relevant historical context.
This entire cycle of prompt adaptation and forecast refinement happens dynamically at test-time, meaning no changes are made to the LLM’s internal weights or training. This offers a practical and cost-effective alternative to traditional model tuning.
Beyond Automated Refinement: Architected Strategy Prompts (ASPs)
The researchers also explored the upper limits of performance with carefully designed prompts, introducing what they call Architected Strategy Prompts (ASPs). These are specialized prompts that include directives for specific analytical procedures or encourage particular cognitive approaches from the LLM. While FLAIRR-TS automates prompt refinement, ASPs demonstrate the potential when manual, strategy-driven design is employed for prompt improvement.
Demonstrated Performance
Experiments conducted on various benchmark datasets, including ETT, Electricity, Traffic, Weather, and ILINet, show that FLAIRR-TS consistently improves forecasting accuracy. It outperforms static, domain-agnostic prompting methods and non-iterative retrieval-augmented baselines. In many scenarios, FLAIRR-TS and ASPs even rival the performance of specialized, hand-tuned prompts and traditional supervised models.
An ablation study further confirmed the individual contributions of both retrieval and iterative refinement. Both components independently reduce forecasting errors, and their combination in FLAIRR-TS delivers the lowest Mean Absolute Error (MAE) across different LLM backbones, including Gemini 2.5 Pro, Gemini 2 Flash, and DeepSeek-V3, highlighting the architecture-agnostic nature of these gains.
Also Read:
- pyFAST: A New PyTorch Framework for Advanced Time Series Analysis with Complex Data
- Navigating the Landscape of AI Agents: Methods and Real-World Applications
Conclusion
FLAIRR-TS represents a significant step forward in time series forecasting with LLMs. By providing a systematic and automated process for refining prompts through an agentic, feedback-driven system, it substantially reduces the burden of manual prompt engineering. This framework offers a scalable pathway to unlock the full potential of LLMs for accurate and adaptable forecasting across diverse datasets and horizons. For more details, you can read the full research paper: FLAIRR-TS – Forecasting LLM-Agents with Iterative Refinement and Retrieval for Time Series.
