Towards A Virtual Assistant That Can Be Taught New Tasks In Any Domain By Its End-Users
This addresses the challenge of enabling end-users to easily customize virtual assistants for arbitrary tasks, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of building an instructible virtual assistant that end-users can teach new tasks via natural language commands and demonstrations, without programming or domain-specific knowledge, achieving teaching times of less than a minute per task in usability studies.
The challenge stated in the title can be divided into two main problems. The first problem is to reliably mimic the way that users interact with user interfaces. The second problem is to build an instructible agent, i.e. one that can be taught to execute tasks expressed as previously unseen natural language commands. This paper proposes a solution to the second problem, a system we call Helpa. End-users can teach Helpa arbitrary new tasks whose level of complexity is similar to the tasks available from today's most popular virtual assistants. Teaching Helpa does not involve any programming. Instead, users teach Helpa by providing just one example of a command paired with a demonstration of how to execute that command. Helpa does not rely on any pre-existing domain-specific knowledge. It is therefore completely domain-independent. Our usability study showed that end-users can teach Helpa many new tasks in less than a minute each, often much less.