Multi-Modal Camera-Based Detection of Vulnerable Road Users
This work addresses a critical safety issue for vulnerable road users in traffic, though it is incremental as it builds on existing YOLOv8 with multimodal data and standard techniques.
The paper tackles the problem of detecting vulnerable road users (VRUs) like pedestrians and cyclists in challenging conditions by developing a multimodal framework that integrates RGB and thermal infrared imaging with a fine-tuned YOLOv8 model. Results show that thermal models achieve the highest precision, and RGB-to-thermal augmentation boosts recall, demonstrating improved VRU safety at intersections.
Vulnerable road users (VRUs) such as pedestrians, cyclists, and motorcyclists represent more than half of global traffic deaths, yet their detection remains challenging in poor lighting, adverse weather, and unbalanced data sets. This paper presents a multimodal detection framework that integrates RGB and thermal infrared imaging with a fine-tuned YOLOv8 model. Training leveraged KITTI, BDD100K, and Teledyne FLIR datasets, with class re-weighting and light augmentations to improve minority-class performance and robustness, experiments show that 640-pixel resolution and partial backbone freezing optimise accuracy and efficiency, while class-weighted losses enhance recall for rare VRUs. Results highlight that thermal models achieve the highest precision, and RGB-to-thermal augmentation boosts recall, demonstrating the potential of multimodal detection to improve VRU safety at intersections.