SEPose: A Synthetic Event-based Human Pose Estimation Dataset for Pedestrian Monitoring
This provides a dataset for pedestrian monitoring systems to improve safety in challenging conditions, but it is incremental as it builds on existing simulators and models.
The authors tackled the lack of event-based data for pedestrian pose estimation by creating SEPose, a synthetic dataset with 350K annotated pedestrians, and showed it enables sim-to-real generalization with existing models.
Event-based sensors have emerged as a promising solution for addressing challenging conditions in pedestrian and traffic monitoring systems. Their low-latency and high dynamic range allow for improved response time in safety-critical situations caused by distracted walking or other unusual movements. However, the availability of data covering such scenarios remains limited. To address this gap, we present SEPose -- a comprehensive synthetic event-based human pose estimation dataset for fixed pedestrian perception generated using dynamic vision sensors in the CARLA simulator. With nearly 350K annotated pedestrians with body pose keypoints from the perspective of fixed traffic cameras, SEPose is a comprehensive synthetic multi-person pose estimation dataset that spans busy and light crowds and traffic across diverse lighting and weather conditions in 4-way intersections in urban, suburban, and rural environments. We train existing state-of-the-art models such as RVT and YOLOv8 on our dataset and evaluate them on real event-based data to demonstrate the sim-to-real generalization capabilities of the proposed dataset.