TLDR: CATS-Linear is a novel time series forecasting model that enhances linear models through two main innovations: Classification Auxiliary Channel-Independence (CACI) and an improved Trend-Seasonal Linear (TSLinear) decomposition. CACI dynamically classifies time series instances and routes them to dedicated predictors based on prediction error, reducing complexity and improving performance. TSLinear refines seasonal and trend component processing using complex-domain projections and a decoupling-recoupling framework. The model achieves state-of-the-art accuracy with fixed hyperparameters, demonstrating superior stability, efficiency, and reduced parameter count compared to existing methods.
Time series forecasting, which involves predicting future values based on historical data, is a critical task in many fields, from finance to weather prediction. Recent advancements have shown that surprisingly simple linear models can achieve forecasting performance comparable to much more complex deep learning architectures. However, methods to further enhance these linear models have remained largely unexplored.
A new research paper introduces CATS-Linear: Classification Auxiliary Linear Model for Time Series Forecasting, a novel approach that significantly boosts the accuracy and efficiency of linear models. The core idea behind CATS-Linear is the hypothesis that different segments or instances within a time series might follow distinct underlying linear patterns. Instead of applying a single model to all data, CATS-Linear intelligently routes these instances to specialized predictors.
Understanding Channel Designs in Time Series Forecasting
Before diving deeper into CATS-Linear, it’s helpful to understand how different models handle multiple features in time series data. Traditionally, there are a few main approaches:
- Channel-Mixing (CM): This method treats all features of a time series as a single, interconnected unit, learning dependencies between them. While it can capture complex relationships, it often leads to a large number of parameters, increasing the risk of overfitting and high memory consumption, especially with high-dimensional data.
- Channel-Independence (CI): In contrast, this approach trains a separate, independent model for each feature. This makes it more robust but can still be memory-intensive when dealing with many features, as it requires numerous individual models.
- One-Channel (OC): This is the simplest method, ignoring feature differences entirely and training a single univariate model for all features. Surprisingly, for some datasets, this can even outperform channel-independent methods.
Introducing Classification Auxiliary Channel-Independence (CACI)
CATS-Linear proposes a new framework called Classification Auxiliary Channel-Independence (CACI) that moves beyond the traditional “feature-as-channel” paradigm. CACI dynamically classifies time series instances into distinct categories using a classifier. Each category is then routed to its own dedicated predictor. This dynamic routing allows the model to adapt to the heterogeneous linear mappings that different time series instances might exhibit.
What makes CACI particularly innovative is its error-supervised design. During training, the system assigns each time series instance to the category whose corresponding predictor yields the minimal prediction error. These error-derived labels are then used to train the classifier. This ensures that the classification is directly relevant to improving prediction accuracy, unlike other clustering methods that might group series in ways not optimal for forecasting. This design also significantly reduces computational complexity from being dependent on the number of features (D) to a constant (O(1)).
Enhanced Trend-Seasonal Decomposition with TSLinear
CATS-Linear also refines the classic trend-seasonal decomposition, a technique that separates a time series into its underlying trend and seasonal components to improve forecasting accuracy. While previous models like DLinear applied linear mappings to these components separately, CATS-Linear introduces a more sophisticated approach:
- Seasonal Components: To better capture periodicity, seasonal components are transformed into complex numbers, leveraging their angular periodicity. Linear transformations are then performed in this complex domain, and the results are converted back to real seasonal predictions.
- Trend Components: The trend components are decoupled into individual time-step states using an exponential smoothing method. Linear mappings are applied to these states, and then they are recoupled to form the final trend predictions. This decoupling and recoupling process helps in generating smoother and more accurate trend forecasts.
This enhanced decomposition framework, combined with the CACI classifier, forms the integrated CATS-Linear system.
Performance and Efficiency
Extensive experiments validate the effectiveness of CATS-Linear. It achieves state-of-the-art accuracy, often comparable to or even surpassing baselines that have been fine-tuned with extensive hyperparameter searching. Crucially, CATS-Linear achieves these results with fixed hyperparameters across all experiments, demonstrating exceptional stability and ease of use.
Compared to other models, CATS-Linear shows significant reductions in Mean Squared Error (MSE). For instance, it reduces MSE by approximately 10% compared to DLinear and by an average of 8% against the runner-up TimeMixer in unified hyperparameter comparisons. Beyond accuracy, CATS-Linear is also highly efficient, boasting the lowest parameter count, Multiply-Accumulate Operations (MACs), and inference time among several leading models. The CACI framework alone significantly reduces the parameter count and inference time for models like DLinear.
An ablation study confirmed the critical role of each component: removing CACI or the enhanced TSLinear led to noticeable performance degradation. The study also showed that CACI is a generalizable method, improving the performance of other models like PatchTST and TiDE when integrated.
Also Read:
- In-Context Learning Tackles Non-Stationary MIMO Equalization
- Unpacking Sequence Model Behavior Through Eigenvalue Analysis
Conclusion
CATS-Linear represents a significant step forward in time series forecasting. By introducing the Classification Auxiliary Channel-Independence (CACI) framework and refining trend-seasonal decomposition, it offers an efficient, accurate, and stable linear model that can achieve top-tier performance without the need for extensive hyperparameter tuning. This makes it a powerful tool for a wide range of forecasting applications. You can read the full research paper here: CATS-Linear: Classification Auxiliary Linear Model for Time Series Forecasting.
