Traj-Transformer: Diffusion Models with Transformer for GPS Trajectory Generation
This work addresses data collection costs and privacy concerns in spatiotemporal data mining for applications like simulating human decision-making, but it is incremental as it improves upon existing diffusion models by replacing the backbone architecture.
The paper tackles the problem of generating realistic GPS trajectories by addressing deviations and loss of fine-grained details in existing diffusion models that use convolution-based architectures, and it demonstrates that the proposed Trajectory Transformer significantly enhances generation quality and alleviates these issues on two real-world datasets.
The widespread use of GPS devices has driven advances in spatiotemporal data mining, enabling machine learning models to simulate human decision making and generate realistic trajectories, addressing both data collection costs and privacy concerns. Recent studies have shown the promise of diffusion models for high-quality trajectory generation. However, most existing methods rely on convolution based architectures (e.g. UNet) to predict noise during the diffusion process, which often results in notable deviations and the loss of fine-grained street-level details due to limited model capacity. In this paper, we propose Trajectory Transformer, a novel model that employs a transformer backbone for both conditional information embedding and noise prediction. We explore two GPS coordinate embedding strategies, location embedding and longitude-latitude embedding, and analyze model performance at different scales. Experiments on two real-world datasets demonstrate that Trajectory Transformer significantly enhances generation quality and effectively alleviates the deviation issues observed in prior approaches.