TLDR: This research paper explores the application of Time Series Foundation Models (TSFMs) within a Conformal Prediction (CP) framework for time series forecasting. It demonstrates that TSFMs, due to their zero-shot capabilities and superior predictive accuracy, provide more reliable and stable prediction intervals, especially in data-constrained scenarios, outperforming traditional forecasting methods.
Accurate forecasting of time series data is vital for making informed decisions across many fields, from finance and healthcare to climate science and retail. Traditionally, this has been tackled using a variety of methods, including classical statistical models like ARIMA, tree-based models such as XGBoost, and modern deep learning architectures.
Recently, a new class of models known as Foundation Models (FMs) has emerged, showing remarkable success in areas like Natural Language Processing and Computer Vision. These large-scale models are pre-trained on vast and diverse datasets, allowing them to learn a wide range of patterns and representations. This enables them to perform specific tasks with little to no additional training, a capability often referred to as “zero-shot” prediction.
The research paper, “Foundation models for time series forecasting: Application in conformal prediction,” explores the potential of these Time Series Foundation Models (TSFMs) in the context of Conformal Prediction (CP). Conformal Prediction is a powerful statistical method used to quantify uncertainty in forecasts. It provides prediction intervals—a range within which the true value is expected to fall with a specified confidence level (e.g., 90% of the time).
A key requirement for Conformal Prediction is a “calibration set,” a separate portion of data used to assess how well the model’s predictions align with actual outcomes. This allows CP to adjust the width of the prediction intervals. However, when data is scarce, partitioning an already small dataset into training and calibration subsets can limit the model’s performance and the accuracy of the calibration.
This is where TSFMs offer a significant advantage. Because of their zero-shot prediction capabilities, TSFMs do not require a large amount of data for training. This means that in a Conformal Prediction setup, most of the available data can be allocated to the calibration process. The study compared TSFMs like Amazon Chronos, Lag-Llama, and Google’s TimesFM with traditional methods on various datasets, including ERCOT, NN5 Daily, NN5 Weekly, and M3 Monthly.
Also Read:
- Unveiling the Continuous Nature of Time Series with NeuTSFlow
- A New Framework for Time Series Forecasting: Bridging Time and Frequency Domains
Key Findings
The findings highlight two main benefits of using TSFMs with Conformal Prediction:
- Reliability with Limited Data: When the amount of data is limited, TSFMs provide more reliable prediction intervals than classic models. This is due to their superior predictive accuracy, which stems from their extensive pre-training.
- Stable Calibration: The calibration process becomes more stable because a larger portion of the available data can be used for calibration. The less data available, the more pronounced these benefits become, as traditional models typically need substantial data for effective training.
The research indicates that TSFMs, particularly the larger models like TimesFM and the Chronos family, consistently achieve better prediction accuracy (lower MASE) and narrower, more meaningful prediction intervals (lower MSIW). This holds true across different data frequencies and forecasting horizons. While some TSFMs occasionally showed slightly lower coverage rates than the target, their overall prediction quality was superior, especially when other models produced overly wide and less useful intervals.
Furthermore, the study notes that TSFMs are efficient in terms of inference time, often comparable to or even faster than traditional statistical models, especially with optimized versions like Chronos Bolt. The paper also observed a positive correlation between model size and performance among TSFMs, suggesting that larger TSFMs tend to yield better results, indicating that the field is still evolving and scaling rules are not yet at a plateau.
In conclusion, this research underscores the significant potential of foundation models in enhancing the reliability of conformal prediction in time series applications, particularly in scenarios where data is a constraint. For more details, you can refer to the full research paper: Foundation models for time series forecasting: Application in conformal prediction.
