Boyi Song

Ruijiao Luo, Chao Huang, Yuntao Peng et al.

In human-robot collaboration (HRC), human trust in the robot is the human expectation that a robot executes tasks with desired performance. A higher-level trust increases the willingness of a human operator to assign tasks, share plans, and reduce the interruption during robot executions, thereby facilitating human-robot integration both physically and mentally. However, due to real-world disturbances, robots inevitably make mistakes, decreasing human trust and further influencing collaboration. Trust is fragile and trust loss is triggered easily when robots show incapability of task executions, making the trust maintenance challenging. To maintain human trust, in this research, a trust repair framework is developed based on a human-to-robot attention transfer (H2R-AT) model and a user trust study. The rationale of this framework is that a prompt mistake correction restores human trust. With H2R-AT, a robot localizes human verbal concerns and makes prompt mistake corrections to avoid task failures in an early stage and to finally improve human trust. User trust study measures trust status before and after the behavior corrections to quantify the trust loss. Robot experiments were designed to cover four typical mistakes, wrong action, wrong region, wrong pose, and wrong spatial relation, validated the accuracy of H2R-AT in robot behavior corrections; a user trust study with $252$ participants was conducted, and the changes in trust levels before and after corrections were evaluated. The effectiveness of the human trust repairing was evaluated by the mistake correction accuracy and the trust improvement.

Boyi Song, Yuntao Peng, Ruijiao Luo et al.

Due to real-world dynamics and hardware uncertainty, robots inevitably fail in task executions, resulting in undesired or even dangerous executions. In order to avoid failures and improve robot performance, it is critical to identify and correct abnormal robot executions at an early stage. However, due to limited reasoning capability and knowledge storage, it is challenging for robots to self-diagnose and -correct their own abnormality in both planning and executing. To improve robot self diagnosis capability, in this research a novel human-to-robot attention transfer (\textit{\textbf{H2R-AT}}) method was developed to identify robot manipulation errors by leveraging human instructions. \textit{\textbf{H2R-AT}} was developed by fusing attention mapping mechanism into a novel stacked neural networks model, transferring human verbal attention into robot visual attention. With the attention transfer, a robot understands \textit{what} and \textit{where} human concerns are to identify and correct abnormal manipulations. Two representative task scenarios: ``serve water for a human in a kitchen" and ``pick up a defective gear in a factory" were designed in a simulation framework CRAIhri with abnormal robot manipulations; and $252$ volunteers were recruited to provide about 12000 verbal reminders to learn and test \textit{\textbf{H2R-AT}}. The method effectiveness was validated by the high accuracy of $73.68\%$ in transferring attention, and the high accuracy of $66.86\%$ in avoiding grasping failures.