Automated capture and delivery of assistive task guidance with an eyewear computer: The GlaciAR system
This work addresses the difficult authoring problem in guidance systems for users needing assistive task support, representing an incremental step toward enhancing natural abilities with automated, on-board technology.
The authors tackled the problem of scaling up task guidance systems by developing GlaciAR, a mixed reality system that automatically captures and delivers assistive video guides using an eyewear computer, enabling users to complete three tasks with minimal invasive guidance.
In this paper we describe and evaluate a mixed reality system that aims to augment users in task guidance applications by combining automated and unsupervised information collection with minimally invasive video guides. The result is a self-contained system that we call GlaciAR (Glass-enabled Contextual Interactions for Augmented Reality), that operates by extracting contextual interactions from observing users performing actions. GlaciAR is able to i) automatically determine moments of relevance based on a head motion attention model, ii) automatically produce video guidance information, iii) trigger these video guides based on an object detection method, iv) learn without supervision from observing multiple users and v) operate fully on-board a current eyewear computer (Google Glass). We describe the components of GlaciAR together with evaluations on how users are able to use the system to achieve three tasks. We see this work as a first step toward the development of systems that aim to scale up the notoriously difficult authoring problem in guidance systems and where people's natural abilities are enhanced via minimally invasive visual guidance.