TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments
This addresses the need for scalable, annotated data in intelligent vehicle systems, though it appears incremental by building on existing datasets.
The paper tackles the problem of generating diverse, high-quality synthetic data for road scene understanding by proposing a pipeline that transforms existing datasets into photorealistic virtual environments, demonstrating improvements in semantic segmentation metrics on Cityscapes and KITTI-STEP datasets.
High-quality structured data with rich annotations are critical components in intelligent vehicle systems dealing with road scenes. However, data curation and annotation require intensive investments and yield low-diversity scenarios. The recently growing interest in synthetic data raises questions about the scope of improvement in such systems and the amount of manual work still required to produce high volumes and variations of simulated data. This work proposes a synthetic data generation pipeline that utilizes existing datasets, like nuScenes, to address the difficulties and domain-gaps present in simulated datasets. We show that using annotations and visual cues from existing datasets, we can facilitate automated multi-modal data generation, mimicking real scene properties with high-fidelity, along with mechanisms to diversify samples in a physically meaningful way. We demonstrate improvements in mIoU metrics by presenting qualitative and quantitative experiments with real and synthetic data for semantic segmentation on the Cityscapes and KITTI-STEP datasets. All relevant code and data is released on github (https://github.com/shubham1810/trove_toolkit).