Supporting User Autonomy with Multimodal Fusion to Detect when a User Needs Assistance from a Social Robot
This work is significant for improving the ability of assistive robots to provide timely and accurate help in task settings, thereby maintaining user autonomy.
This paper addresses the challenge of a social robot detecting when a user needs assistance while preserving user autonomy. The authors developed a late fusion model, combining four independent high-precision, low-recall models (mutual gaze, task, confirmatory gaze, and lexical), which significantly outperformed the individual models in accuracy.
It is crucial for any assistive robot to prioritize the autonomy of the user. For a robot working in a task setting to effectively maintain a user's autonomy it must provide timely assistance and make accurate decisions. We use four independent high-precision, low-recall models, a mutual gaze model, task model, confirmatory gaze model, and a lexical model, that predict a user's need for assistance. Improving upon our four independent models, we used a sliding window method and a random forest classification algorithm to capture temporal dependencies and fuse the independent models with a late fusion approach. The late fusion approach strongly outperforms all four of the independent models providing a more wholesome approach with greater accuracy to better assist the user while maintaining their autonomy. These results can provide insight into the potential of including additional modalities and utilizing assistive robots in more task settings.