Looking at Hands in Autonomous Vehicles: A ConvNet Approach using Part Affinity Fields
This addresses safety monitoring in autonomous driving, but it is incremental as it adapts an existing method (OpenPose) to a specific domain.
The paper tackles the problem of locating driver and passenger hands in autonomous vehicles to monitor readiness for takeover requests, achieving at least 95% detection performance on joint localization and arm-angle estimation while running at 40 fps in real-time.
In the context of autonomous driving, where humans may need to take over in the event where the computer may issue a takeover request, a key step towards driving safety is the monitoring of the hands to ensure the driver is ready for such a request. This work, focuses on the first step of this process, which is to locate the hands. Such a system must work in real-time and under varying harsh lighting conditions. This paper introduces a fast ConvNet approach, based on the work of original work of OpenPose for full body joint estimation. The network is modified with fewer parameters and retrained using our own day-time naturalistic autonomous driving dataset to estimate joint and affinity heatmaps for driver & passenger's wrist and elbows, for a total of 8 joint classes and part affinity fields between each wrist-elbow pair. The approach runs real-time on real-world data at 40 fps on multiple drivers and passengers. The system is extensively evaluated both quantitatively and qualitatively, showing at least 95% detection performance on joint localization and arm-angle estimation.