Spiking Neural Networks for Early Prediction in Human Robot Collaboration
It addresses the need for proactive robot assistance in collaborative tasks like surgery to improve team efficiency, representing a domain-specific incremental advance.
This paper tackles the problem of early turn-taking prediction in human-robot collaboration by introducing TTSNet, which uses multimodal cues to predict intentions, achieving an F1 score of 0.683 with only 10% of action completion and outperforming state-of-the-art algorithms and human performance under partial observation.
This paper introduces the Turn-Taking Spiking Neural Network (TTSNet), which is a cognitive model to perform early turn-taking prediction about human or agent's intentions. The TTSNet framework relies on implicit and explicit multimodal communication cues (physical, neurological and physiological) to be able to predict when the turn-taking event will occur in a robust and unambiguous fashion. To test the theories proposed, the TTSNet framework was implemented on an assistant robotic nurse, which predicts surgeon's turn-taking intentions and delivers surgical instruments accordingly. Experiments were conducted to evaluate TTSNet's performance in early turn-taking prediction. It was found to reach a F1 score of 0.683 given 10% of completed action, and a F1 score of 0.852 at 50% and 0.894 at 100% of the completed action. This performance outperformed multiple state-of-the-art algorithms, and surpassed human performance when limited partial observation is given (< 40%). Such early turn-taking prediction capability would allow robots to perform collaborative actions proactively, in order to facilitate collaboration and increase team efficiency.