TLDR: MillGNN is a novel graph neural network designed for multi-variate time series forecasting. It addresses the limitation of existing methods by learning and utilizing lead-lag dependencies at multiple grouping scales, capturing how variables and groups of variables influence each other over time with dynamic and decaying effects. Through its scale-specific lead-lag graph learning and hierarchical message passing modules, MillGNN achieves superior forecasting accuracy on 11 real-world datasets compared to 16 state-of-the-art methods, particularly excelling in scenarios with strong spatial correlations.
Multi-variate time series (MTS) forecasting is a critical task across many fields, from managing traffic to predicting weather patterns. These systems involve numerous interacting variables, and accurately predicting their future states is a significant challenge. Traditional methods have made strides in understanding how individual variables change over time and how different variables influence each another at the same moment. However, a crucial aspect often overlooked is the concept of ‘lead-lag dependencies’ at multiple grouping scales.
Imagine a complex system like a power grid or city traffic. A small event, like a voltage drop at a single transformer, might quickly affect nearby sensors. This is a fine-grained lead-lag dependency. But a larger event, such as a regional blackout, could trigger cascading effects across entire city districts over much longer periods. This represents a coarse-grained lead-lag dependency between larger groups of variables. Existing forecasting models often struggle to capture these hierarchical, multi-scale lead-lag effects, limiting their ability to truly understand and predict complex system behaviors.
To address this gap, researchers have introduced MillGNN, a new method based on graph neural networks. MillGNN is designed to learn these multi-scale lead-lag dependencies for MTS forecasting, providing a more comprehensive understanding of how variables and groups of variables influence each other over time, including how these influences might decay.
How MillGNN Works
MillGNN introduces two key innovations:
1. Scale-Specific Lead-Lag Graph Learning (SiLL-GL) Module: This module is responsible for identifying lead-lag relationships at each grouping scale. It starts by using a technique called Fast Fourier Transform (FFT)-based cross-correlation to find initial lead-lag connections. Think of cross-correlation as a way to see how much two time series are related, and at what time delay one might follow the other. MillGNN then refines these connections using a ‘decay-aware attention mechanism.’ This mechanism dynamically adjusts the strength of these lead-lag links, considering how their influence might diminish over time or be periodically revived by cyclical patterns. This allows the model to adapt to the unique dynamics and decaying nature of dependencies at different scales, offering both statistical insight and data-driven flexibility.
2. Hierarchical Lead-Lag Message Passing (HiLL-MP) Module: Once the multi-scale lead-lag relationships are identified, this module efficiently propagates these influences. It passes fine-grained lead-lag messages between similar variables or groups within the same scale. Simultaneously, it progressively passes coarser-grained messages between dissimilar groups across different scales. This structured approach ensures that the model captures lead-lag effects with mixed granularities, balancing thoroughness with computational efficiency. For instance, a target variable might receive direct messages from closely related variables (intra-scale) and also aggregated messages from larger parent groups (inter-scale).
Performance and Insights
MillGNN was rigorously tested on 11 real-world datasets, including those for electricity consumption, weather, and traffic, and compared against 16 state-of-the-art forecasting methods. The experimental results consistently demonstrated MillGNN’s superior performance for both long-term and short-term MTS forecasting.
The model showed particularly strong gains in datasets with significant spatial correlations, such as traffic and air quality (China-AQI) data. This is likely because the spread of traffic flow or air pollution is heavily influenced by distance, creating natural hierarchical propagation patterns that MillGNN is uniquely designed to capture.
Interestingly, the study also revealed that MillGNN performs better on datasets where the lead-lag patterns (lag distributions) remain stable between training and test periods. This provides valuable guidance for its application in real-world scenarios.
Ablation studies confirmed the importance of each of MillGNN’s core components, showing performance drops when multi-scale construction, initial lead-lag graphs, or the decay-aware attention mechanism were removed. The hierarchical message passing module also proved crucial for efficient and effective propagation of lead-lag effects.
In a case study on China AQI prediction, MillGNN successfully detected lead-lag effects between cities like Nantong and Shanghai, and even between larger variate groups, without needing external information like wind direction. This highlights its ability to uncover hidden dynamics purely from the time series data.
Also Read:
- Unlocking Deeper Insights: A New Approach to Forecasting Multivariate Time Series
- TimeCopilot: Unifying Time Series Forecasting with Agentic AI and Large Language Models
Conclusion
MillGNN represents a significant advancement in multi-variate time series forecasting by effectively learning and leveraging multi-scale lead-lag dependencies. By considering how different parts of a system influence each other over time, at both individual and group levels, MillGNN offers a more comprehensive and accurate approach to prediction. Future work aims to explore even more complex lead-lag dependencies and develop more efficient grouping strategies. For more technical details, you can refer to the full research paper here.
