Multimodal Interactive Learning of Primitive Actions
This work addresses the challenge of teaching action concepts to machines for robotics or AI applications, but it appears incremental as it builds on prior demonstration-based methods.
The paper tackles the problem of learning primitive actions from demonstrations by incorporating multimodal human-computer interaction to fine-tune a neural network-based trajectory model, aiming to overcome limitations of previous work with few training samples.
We describe an ongoing project in learning to perform primitive actions from demonstrations using an interactive interface. In our previous work, we have used demonstrations captured from humans performing actions as training samples for a neural network-based trajectory model of actions to be performed by a computational agent in novel setups. We found that our original framework had some limitations that we hope to overcome by incorporating communication between the human and the computational agent, using the interaction between them to fine-tune the model learned by the machine. We propose a framework that uses multimodal human-computer interaction to teach action concepts to machines, making use of both live demonstration and communication through natural language, as two distinct teaching modalities, while requiring few training samples.