Text2LiDAR: Text-guided LiDAR Point Cloud Generation via Equirectangular Transformer
This addresses the problem of expensive and challenging LiDAR data collection in complex traffic environments for researchers and practitioners in autonomous driving, representing a novel method for a known bottleneck.
The paper tackles the challenge of generating high-quality and controllable LiDAR point clouds from text descriptions, proposing Text2LiDAR as an efficient and diverse model that achieves superior performance on datasets like KITTI-360 and nuScenes.
The complex traffic environment and various weather conditions make the collection of LiDAR data expensive and challenging. Achieving high-quality and controllable LiDAR data generation is urgently needed, controlling with text is a common practice, but there is little research in this field. To this end, we propose Text2LiDAR, the first efficient, diverse, and text-controllable LiDAR data generation model. Specifically, we design an equirectangular transformer architecture, utilizing the designed equirectangular attention to capture LiDAR features in a manner with data characteristics. Then, we design a control-signal embedding injector to efficiently integrate control signals through the global-to-focused attention mechanism. Additionally, we devise a frequency modulator to assist the model in recovering high-frequency details, ensuring the clarity of the generated point cloud. To foster development in the field and optimize text-controlled generation performance, we construct nuLiDARtext which offers diverse text descriptors for 34,149 LiDAR point clouds from 850 scenes. Experiments on uncontrolled and text-controlled generation in various forms on KITTI-360 and nuScenes datasets demonstrate the superiority of our approach.