Gesture Matters: Pedestrian Gesture Recognition for AVs Through Skeleton Pose Evaluation
This work addresses the challenge of improving pedestrian-vehicle interactions for autonomous vehicles, though it is incremental as it builds on existing pose estimation methods.
The study tackled the problem of autonomous vehicles struggling to interpret pedestrian gestures by developing a gesture classification framework using 2D pose estimation on real-world video data, achieving a classification accuracy of 87%.
Gestures are a key component of non-verbal communication in traffic, often helping pedestrian-to-driver interactions when formal traffic rules may be insufficient. This problem becomes more apparent when autonomous vehicles (AVs) struggle to interpret such gestures. In this study, we present a gesture classification framework using 2D pose estimation applied to real-world video sequences from the WIVW dataset. We categorise gestures into four primary classes (Stop, Go, Thank & Greet, and No Gesture) and extract 76 static and dynamic features from normalised keypoints. Our analysis demonstrates that hand position and movement velocity are especially discriminative in distinguishing between gesture classes, achieving a classification accuracy score of 87%. These findings not only improve the perceptual capabilities of AV systems but also contribute to the broader understanding of pedestrian behaviour in traffic contexts.