Graph-based Online Monitoring of Train Driver States via Facial and Skeletal Features
This work addresses railway safety by improving online monitoring of train driver states, though it represents an incremental advancement in vision-based monitoring with a novel dataset.
This study tackled the problem of train driver fatigue monitoring by developing an online behavior-based system using a Directed-Graph Neural Network (DGNN) that classifies driver states into alert, not alert, and pathological categories. The system achieved 80.88% accuracy in three-class classification and over 99% accuracy in binary alertness classification when combining facial and skeletal features.
Driver fatigue poses a significant challenge to railway safety, with traditional systems like the dead-man switch offering limited and basic alertness checks. This study presents an online behavior-based monitoring system utilizing a customised Directed-Graph Neural Network (DGNN) to classify train driver's states into three categories: alert, not alert, and pathological. To optimize input representations for the model, an ablation study was performed, comparing three feature configurations: skeletal-only, facial-only, and a combination of both. Experimental results show that combining facial and skeletal features yields the highest accuracy (80.88%) in the three-class model, outperforming models using only facial or skeletal features. Furthermore, this combination achieves over 99% accuracy in the binary alertness classification. Additionally, we introduced a novel dataset that, for the first time, incorporates simulated pathological conditions into train driver monitoring, broadening the scope for assessing risks related to fatigue and health. This work represents a step forward in enhancing railway safety through advanced online monitoring using vision-based technologies.