TLDR: PhaseFormer is a novel deep learning model for time series forecasting that introduces a ‘phase-based’ perspective, focusing on values aligned across cycles rather than traditional ‘patches.’ This approach leads to more stable and lower-dimensional data representations, enabling PhaseFormer to achieve state-of-the-art prediction accuracy with significantly reduced computational costs and parameter counts (over 99.9% less than leading patch-based models). It particularly excels on complex, large-scale datasets, marking a major advancement in efficient and effective time series prediction.
Time series forecasting is a critical tool that helps us make decisions across many fields, from predicting weather and energy use to managing traffic and healthcare. In recent years, deep learning has shown great promise in this area, using its ability to learn complex patterns from historical data to predict future trends.
A key characteristic of much real-world time series data is its periodicity – patterns that repeat over time. Many advanced deep learning methods have tried to leverage this by breaking down sequences into ‘patches’ or segments, which are then processed by sophisticated models. While these patch-based approaches have improved prediction accuracy, they often come with a significant drawback: inefficiency. They tend to have a large number of parameters and high computational costs, making them slow and resource-intensive, especially for large and complex datasets.
The research paper, PHASEFORMER: FROM PATCHES TO PHASES FOR EFFICIENT AND EFFECTIVE TIME SERIES FORECASTING, introduces a new perspective to tackle this challenge. It explains that the inefficiency of patch-level processing stems from the high variability of cycle patterns in real-world data. Factors like new infrastructure affecting traffic or changing work schedules impacting electricity demand can cause these patterns to shift, forcing models to create complex, high-dimensional representations that are computationally expensive.
Introducing the Phase Perspective
To overcome these limitations, the authors propose a novel ‘phase-based’ perspective. Instead of looking at patches (adjacent observations within a local period), PhaseFormer focuses on ‘phase tokens’ – values aligned at the same offset across successive cycles. Imagine looking at the traffic flow at 8 AM every Monday, rather than a continuous block of traffic data for an hour. These phase tokens show much less variability than patch tokens, leading to more efficient and generalizable representations.
The paper provides strong evidence for this. Visualizations show that while patch tokens drift continuously over time, phase tokens form compact and stable clusters. Quantitatively, phase tokens exhibit significantly lower temporal distribution divergence. Furthermore, phase tokens reside in a much lower-dimensional space; just two dimensions can explain over 90% of their variance, compared to more than eleven dimensions needed for patch tokens. This inherent low-dimensionality is a principled basis for building models that are both parameter and computation-efficient.
PhaseFormer: An Efficient and Effective Solution
Building on these insights, the researchers introduce PhaseFormer, a lightweight forecasting model. PhaseFormer works by:
- Aligning and extracting phase tokens from the input sequence and mapping them into a shared low-dimensional latent space.
- Employing a lightweight ‘routing mechanism’ to enable efficient communication across different phases.
- Applying a shared predictor to project these latent representations into forecasts for each phase.
This architecture avoids the computationally expensive full pairwise interactions of traditional self-attention mechanisms by using a set of learnable ‘routers’ to mediate information exchange. This two-step process involves ‘phase-to-router aggregation’ (routers gather information from phases) and ‘router-to-phase distribution’ (routers send aggregated information back to phases), all implemented via efficient cross-attention.
Also Read:
- Glocal Information Bottleneck: A New Approach for Time Series Imputation
- Predicting Rare Events: A Deep Learning Approach to Extreme Value Forecasting
Remarkable Performance and Efficiency
Extensive experiments demonstrate that PhaseFormer achieves state-of-the-art performance across seven benchmark datasets. Notably, it excels on large-scale and complex datasets like Traffic, Electricity, and Weather. For instance, on the Traffic dataset, PhaseFormer surpasses the second-best method, PatchTST, by 6.3% in accuracy. Crucially, PhaseFormer achieves these results with an extraordinary efficiency gain, showing over a 99.9% reduction in both parameter count and computational cost (FLOPs) compared to leading patch-based models like PatchTST and Crossformer.
Ablation studies further confirm the design choices. The model performs optimally with a small number of routers (typically 4 or 8), reinforcing the idea that phase tokens occupy a low-dimensional space. The cross-phase routing layer is shown to be essential for modeling periodic dynamics, outperforming simpler linear mixing or no routing at all, and even being more efficient than full attention.
This work represents a significant step towards truly efficient and effective time series forecasting, offering a practical pathway for building powerful forecasting models without the need for heavy and complex architectures. The code for PhaseFormer is publicly available for the research community.
