TLDR: GeoMAE is a self-supervised learning model designed to accurately forecast spatio-temporal graph data despite significant missing values. It uses a novel input preprocessing module, an attention-based network, and a masking autoencoder-inspired auxiliary task to enhance robustness and generalization, outperforming existing methods on real-world air quality datasets.
Accurate data on crowd flow at Points of Interest (POIs) is crucial for effective urban management, including traffic, public services, and city planning. However, limitations in urban sensing technologies often lead to low-quality data with significant missing values. This makes it challenging to accurately infer crowd flow, especially given the scarcity of labeled data, complex spatio-temporal dependencies among POIs, and correlations with GPS reports.
Similarly, missing values are a common problem in many practical systems, such as traffic forecasting and energy consumption prediction, due to harsh environments and device failures. These missing values negatively impact downstream applications in intelligent cities. Existing solutions often focus on time-series analysis, overlooking dynamic spatial correlations and struggling with complicated and fluctuating missing patterns, which can vary greatly due to maintenance conditions and budgets.
To address these critical challenges, researchers have introduced a novel self-supervised spatio-temporal representation learning model called GeoMAE. This model is specifically designed for predicting spatio-temporal graph data even when values are missing. GeoMAE aims to extract valuable knowledge from incomplete data and enhance the generalization ability of models in real-world scenarios.
How GeoMAE Works
GeoMAE is built upon three main components that work together to handle incomplete data effectively:
The first component is an Input Preprocessing Module. This module is designed to minimize the impact of missing values on the overall data distribution. It uses a random filling strategy for missing data points, combined with a specially designed ‘masking matrix’ called HintSensor. This approach helps the model adapt better to varying rates of missing data, preventing biases that can arise from high missing rates.
Next, GeoMAE employs an Attention-based Spatio-Temporal Forecasting Network (STAFN). This network is crucial for capturing the intricate and dynamic spatio-temporal relationships within the data. STAFN utilizes an Encoder-Decoder architecture, incorporating multiple spatio-temporal attention and spatial attention modules. These attention mechanisms allow the model to learn how different locations and time points influence each other, even with incomplete information.
Finally, GeoMAE includes an Auxiliary Learning Task inspired by Masking AutoEncoders (MAE). To enhance the model’s robustness and generalization, this task generates multiple ‘augmented’ versions of the input data by adding extra masks, simulating various missing patterns and rates. The model then uses a self-supervised loss function to ensure that the representations learned from these augmented samples are consistent with the representation of the original, incomplete data. This process helps the model become more resilient to different missing data scenarios it might encounter in practice.
Also Read:
- STPFormer: A New Approach to Smarter Traffic Prediction
- Advancing Long-Term Spatio-Temporal Prediction with Multiscale Mamba Models
Experimental Validation
The effectiveness of GeoMAE was rigorously tested on a real-world dataset called BJ-Air, which contains air quality data from 35 monitoring stations in Beijing over three years (2015-2017). The experiments simulated various missing rates (25%, 50%, 75%, 90%) and patterns, including ‘point missing’ and ‘block missing’ scenarios.
The results demonstrated that GeoMAE consistently outperformed several established baseline models, such as MLP, LSTM, GRU-D, AGCRN, GWNet, and TriD-MAE. This superior performance highlights GeoMAE’s ability to effectively learn spatio-temporal representations from data with missing values for prediction tasks. The self-supervised auxiliary loss and the innovative input data preprocessing modules were shown to significantly enhance the model’s generalization ability and robustness.
In conclusion, GeoMAE offers a robust solution for spatio-temporal graph forecasting in the presence of missing values, addressing key challenges related to dynamic correlations, diverse missing patterns, and fluctuating data conditions. Its consistent performance across complex missing scenarios makes it a promising advancement for smart city applications. For more technical details, you can refer to the full research paper.
