TLDR: Traj-Transformer is a novel model that uses a transformer architecture for GPS trajectory generation, addressing the limitations of previous convolution-based methods. It introduces a longitude-latitude embedding strategy that significantly improves the preservation of fine-grained street-level details and reduces trajectory deviations. Experiments on real-world datasets show that Traj-Transformer consistently outperforms existing models, even with fewer parameters, and demonstrates strong utility for applications like traffic flow prediction and diverse trajectory generation.
The increasing use of GPS devices has led to significant advancements in understanding and utilizing spatiotemporal data. This data is crucial for various applications, from urban planning and intelligent transportation systems to human mobility analysis and public safety. However, collecting large amounts of real-world GPS trajectory data is expensive and raises privacy concerns. This has driven the need for machine learning models that can generate realistic, human-like trajectories.
Recent research has shown that diffusion models are very promising for creating high-quality trajectories. Yet, most existing methods, like those based on UNet architectures, use convolutions to predict noise during the generation process. These convolution-based models often struggle to capture fine-grained street-level details and can produce trajectories that deviate noticeably from actual road networks, largely due to their limited model capacity.
A new research paper introduces the Traj-Transformer, a novel model designed to overcome these limitations. It employs a transformer backbone for both embedding conditional information and predicting noise during the diffusion process. This approach allows the model to better understand and reproduce the complex patterns of human movement.
Key Innovations of Traj-Transformer
The Traj-Transformer explores two main strategies for embedding GPS coordinates: location embedding (loc-emb) and longitude-latitude embedding (lon-lat-emb). Location embedding treats longitude and latitude together as a single point, while longitude-latitude embedding processes them separately. The researchers found that embedding longitude and latitude independently (lon-lat-emb) consistently leads to better performance. This is because it preserves more detailed spatial information, which is vital for capturing the intricate street-level details in trajectory data.
To efficiently integrate conditional information (like starting and ending points) and diffusion timesteps into the model, Traj-Transformer uses an adaptive layer norm (adaLN) approach. This method is computationally efficient and helps guide the generation process towards desired targets.
Also Read:
- Unlocking Human Movement: A Deep Generative Model for Realistic Mobility Behavior
- HTMformer: Enhancing Time Series Forecasting Accuracy and Efficiency with Hybrid Embeddings
Performance and Results
Experiments were conducted on two large-scale real-world GPS trajectory datasets from major Chinese cities, Chengdu and Xi’an. The results demonstrate that Traj-Transformer significantly improves the quality of generated trajectories and effectively reduces the deviation issues seen in previous approaches. Even with fewer parameters, the Traj-Transformer models, especially those using the lon-lat-emb strategy, consistently outperformed existing convolution-based baselines like Traj-UNet, Geo-UNet, and WaveNet.
Qualitative visualizations further highlight these improvements. While older models often produce trajectories that appear over-smoothed or geometrically inconsistent, particularly in dense urban areas, Traj-Transformer accurately recovers detailed street geometries. The largest configuration of Traj-Transformer with lon-lat-emb achieved the highest fidelity, producing smoother and more realistic trajectories that closely align with actual street networks.
Beyond just generating realistic paths, the utility of the generated data was also assessed for downstream applications, such as traffic flow prediction. Traj-Transformer consistently achieved lower density errors across different time periods, indicating its enhanced capacity to reproduce realistic movement behaviors and support analytical tasks. The model also demonstrated its ability to generate a diverse range of distinct yet realistic trajectories under fixed conditions, showcasing its strength in learning rich, multimodal movement patterns.
In conclusion, Traj-Transformer offers a powerful and efficient solution for GPS trajectory generation. By leveraging a transformer architecture and an effective longitude-latitude embedding strategy, it generates high-quality, realistic trajectories without needing auxiliary road information, simplifying the training process. This work paves the way for future end-to-end transformer-based pipelines in spatiotemporal data generation. You can read the full research paper here.
