ESCAPE - Echo SCraper and ClAssifier of PErsons: A novel tool to facilitate using voice-controlled devices for research
This tool facilitates research using voice-controlled devices for studying participants in naturalistic settings, with potential applications in monitoring diseases, but it is incremental as it builds on existing speaker recognition and data extraction methods.
The authors tackled the problem of extracting and analyzing data from voice-controlled assistants for research by developing ESCAPE, an open-source tool that extracts Amazon Echo interaction data and performs speaker recognition with accuracy using a small labeled dataset, yielding a rich dataset from 3 months of recordings.
Smart devices have become common place in many homes, and these devices can be utilized to provide support for people with mental or physical deficits. Voice-controlled assistants are a class of smart device that collect a large amount of data in the home. In this work we present Echo SCraper and ClAssifier of Persons (ESCAPE), an open source software for the extraction of Amazon Echo interaction data, and speaker recognition on that data. We show that ESCAPE is able to extract data from a voice-controlled assistant and classify with accuracy who is talking, based on a small number of labeled audio data. Using ESCAPE to extract interactions recorded over 3 months in the first author's home yields a rich dataset of transcribed audio recordings. Our results demonstrate that using this software the Amazon Echo can be used to study participants in a naturalistic setting with minimal intrusion. We also discuss the potential for usage of voice-controlled devices together with ESCAPE to understand how diseases affect individuals, and how these data can be used to monitor disease processes in general.