TLDR: This research paper, ‘Foundation Model Forecasts: Form and Function,’ by Alvaro Perez-Diaz et al., argues that the practical utility of Time-Series Foundation Models (TSFMs) depends not just on their accuracy, but critically on the ‘form’ of their forecasts. It categorizes forecasts into point, quantile, parametric, and trajectory ensembles, establishing an expressiveness hierarchy. The paper demonstrates that many operational tasks, especially those involving path-dependent events (like cumulative risk or threshold crossing), require the richer ‘trajectory ensemble’ form, which preserves temporal dependence. It proves that simpler forecast types cannot reliably answer these questions without additional, often unvalidated, assumptions. The authors provide a mapping of six fundamental forecasting tasks to the minimal sufficient forecast types and emphasize the need for task-aligned evaluation metrics, concluding that forecast type is a primary differentiator of practical value.
Time-series foundation models (TSFMs) are rapidly changing how we predict future events, from stock prices to weather patterns. These advanced models often boast impressive accuracy, but a new research paper titled ‘Foundation Model Forecasts: Form and Function’ argues that accuracy alone isn’t enough. The paper, authored by Alvaro Perez-Diaz, James C. Loach, Danielle E. Toutoungi, and Lee Middleton from Senseye, Siemens Digital Industries, highlights a crucial, yet often overlooked, aspect of forecasting: the ‘form’ of the forecast itself. This form dictates what real-world problems a forecast can actually help solve. You can read the full paper here: Foundation Model Forecasts: Form and Function.
Understanding the Different Forms of Forecasts
The researchers categorize forecasts into four main types, each with varying levels of detail and utility:
Point Forecasts: These are the simplest, providing a single predicted value for each future time step. For example, predicting tomorrow’s temperature as 20 degrees Celsius. While easy to understand, they offer no information about uncertainty or the range of possible outcomes.
Quantile Forecasts: These go a step further by providing a range of possible values at different probability levels for each future time step. For instance, predicting that tomorrow’s temperature will be between 18 and 22 degrees Celsius with 90% confidence. They give a sense of uncertainty for individual points in time.
Parametric Forecasts: Similar to quantile forecasts, these also describe uncertainty for each future time step, but they do so by providing the parameters of a specific probability distribution (like the mean and variance of a Gaussian distribution). This allows for more detailed calculations about individual time steps.
Trajectory Ensembles: This is the most comprehensive and expressive form. Instead of just predicting individual points or distributions for each step, a trajectory ensemble generates multiple complete ‘paths’ or scenarios of how the future might unfold. These paths preserve the temporal dependence, meaning they show how events at one time step influence subsequent ones. Imagine not just predicting tomorrow’s temperature, but generating 100 different plausible weather sequences for the entire week, showing how temperature, humidity, and wind might interact.
Why the Form Matters More Than You Think
The paper emphasizes an ‘expressiveness hierarchy’: trajectory ensembles are the most expressive, followed by parametric and quantile forecasts, and then point forecasts. This hierarchy is critical because it determines what questions a forecast can answer without making additional, potentially risky, assumptions.
Converting a more expressive forecast type to a simpler one is straightforward. For example, you can easily get a point forecast (like the average) or quantile forecasts from a trajectory ensemble. However, the reverse is much harder. Going from simpler forms (like point or quantile forecasts) to a trajectory ensemble requires making explicit assumptions about how different time steps are related (their ‘temporal dependence’). Without these assumptions, you lose crucial information about how events co-vary over time.
A key mathematical finding in the paper is that per-step marginal distributions (what quantile and parametric forecasts provide) cannot uniquely determine probabilities for events that depend on the entire path of future outcomes. This means that if your operational task involves understanding sequences of events, marginal forecasts alone are insufficient.
Matching Forecasts to Real-World Tasks
The researchers map six fundamental forecasting tasks to the minimal forecast type required to support them effectively:
1. Pointwise Prediction Intervals: For questions like ‘what will the value be at each future time and what is my confidence?’, quantile or parametric forecasts are sufficient, as they provide per-step uncertainty.
2. Pathwise Forecast Bands: If you need a band that simultaneously covers the entire forecast horizon (e.g., ‘give me a band containing the next 24 hours with 95% confidence’), trajectory ensembles are ideal. Simpler forms can be used but require additional assumptions or complex post-processing.
3. Event and Tail Probabilities: For risk management questions like ‘what is the probability our cumulative loss exceeds a threshold?’, trajectory ensembles are essential because they capture the joint behavior of variables over time. Parametric forecasts can work with assumptions about temporal dependence, but point forecasts are unsuitable.
4. Threshold Crossing and Persistence: Tasks such as ‘when will vibration first exceed my machine’s safety limit?’ or ‘how long will pollution stay above the legal threshold?’ require understanding sequences of events. Trajectory ensembles directly provide this, while simpler forms struggle to quantify uncertainty for such path-dependent events.
5. Window Aggregates: For planning tasks like ‘what is the expected demand during the holiday week?’, trajectory ensembles can directly sum up values across paths to provide a distribution of the aggregate. Simpler forecasts can provide a sum of means but lack the full distribution of the aggregate without further assumptions.
6. Scenario Generation and Ranking: When strategic planning needs concrete future scenarios to stress-test decisions (e.g., ‘generate plausible metro load paths to test capacity limits’), trajectory ensembles are the only native solution, as they explicitly produce multiple plausible future paths.
Evaluating Forecasts Beyond Simple Accuracy
The paper also stresses that evaluation metrics must align with the forecast type and the intended task. Metrics like Mean Absolute Error (MAE) or Mean Squared Error (MSE) are fine for point forecasts, but they tell you nothing about uncertainty. For probabilistic forecasts, metrics like Continuous Ranked Probability Score (CRPS) assess marginal accuracy. However, for path-dependent tasks, specialized metrics like the Energy Score or Variogram Score are needed to evaluate whether the forecast accurately captures temporal dependence and joint distributions.
Also Read:
- Probabilistic Kolmogorov-Arnold Networks Enhance Satellite Traffic Forecasting
- AI-Powered Electricity Price Forecasting for Dynamic Markets
Conclusion: The Practical Utility of Forecast Form
The core message is clear: while TSFMs are achieving impressive accuracy, their true practical value is determined by the ‘form’ of their output. Trajectory ensembles offer the most flexibility for a wide range of operational tasks, especially those involving path-dependent events. Developers of TSFMs should consider producing richer forecast types, and practitioners should carefully select models based on the specific questions they need to answer, not just their reported accuracy scores. When multiple models show similar accuracy, the forecast type becomes the primary differentiator of practical utility.
