MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?
This addresses data acquisition and labeling challenges for researchers and practitioners in computer vision, particularly in crowded public environments, but it is incremental as it applies an existing method (synthetic data generation) to a new domain.
The paper tackles the problem of data privacy and annotation costs in video pedestrian detection and tracking by generating MOTSynth, a large synthetic dataset using a rendering game engine. The result shows that MOTSynth can effectively replace real data for tasks like pedestrian detection, re-identification, segmentation, and tracking, though specific performance numbers are not provided in the abstract.
Deep learning-based methods for video pedestrian detection and tracking require large volumes of training data to achieve good performance. However, data acquisition in crowded public environments raises data privacy concerns -- we are not allowed to simply record and store data without the explicit consent of all participants. Furthermore, the annotation of such data for computer vision applications usually requires a substantial amount of manual effort, especially in the video domain. Labeling instances of pedestrians in highly crowded scenarios can be challenging even for human annotators and may introduce errors in the training data. In this paper, we study how we can advance different aspects of multi-person tracking using solely synthetic data. To this end, we generate MOTSynth, a large, highly diverse synthetic dataset for object detection and tracking using a rendering game engine. Our experiments show that MOTSynth can be used as a replacement for real data on tasks such as pedestrian detection, re-identification, segmentation, and tracking.