InteRACT: Transformer Models for Human Intent Prediction Conditioned on Robot Actions
This work addresses the chicken-or-egg problem in human-robot collaboration for manipulation tasks, offering a conditional intent prediction model that could enhance robot adaptability, though it is incremental as it builds on prior methods by leveraging transfer learning.
The paper tackles the problem of predicting human intents in collaborative human-robot manipulation by addressing the inter-dependency between human intents and robot actions, proposing InteRACT, a transformer model that pre-trains on human-human data and fine-tunes on human-robot data, which improves over marginal baselines in real-world tasks.
In collaborative human-robot manipulation, a robot must predict human intents and adapt its actions accordingly to smoothly execute tasks. However, the human's intent in turn depends on actions the robot takes, creating a chicken-or-egg problem. Prior methods ignore such inter-dependency and instead train marginal intent prediction models independent of robot actions. This is because training conditional models is hard given a lack of paired human-robot interaction datasets. Can we instead leverage large-scale human-human interaction data that is more easily accessible? Our key insight is to exploit a correspondence between human and robot actions that enables transfer learning from human-human to human-robot data. We propose a novel architecture, InteRACT, that pre-trains a conditional intent prediction model on large human-human datasets and fine-tunes on a small human-robot dataset. We evaluate on a set of real-world collaborative human-robot manipulation tasks and show that our conditional model improves over various marginal baselines. We also introduce new techniques to tele-operate a 7-DoF robot arm and collect a diverse range of human-robot collaborative manipulation data, which we open-source.