Opportunistic and Context-aware Affect Sensing on Smartphones: The Concept, Challenges and Opportunities
This work addresses the problem of reducing biases in affect sensing for users by moving from lab settings to real-world smartphone applications, though it is incremental as it builds on existing technologies and identifies rather than solves barriers.
The paper tackles the challenge of implementing opportunistic and context-aware affect sensing on smartphones by leveraging low-power DSP and GPU technologies to capture spontaneous facial expressions and voice, while identifying key barriers and potential solutions for this approach.
Opportunistic affect sensing offers unprecedented potential for capturing spontaneous affect ubiquitously, obviating biases inherent in the laboratory setting. Facial expression and voice are two major affective displays, however most affect sensing systems on smartphone avoid them due to extensive power requirement. Encouragingly, due to the recent advent of low-power DSP (Digital Signal Processing) co-processor and GPU (Graphics Processing Unit) technology, audio and video sensing are becoming more feasible. To properly evaluate opportunistically captured facial expression and voice, contextual information about the dynamic audio-visual stimuli needs to be inferred. This paper discusses recent advances of affect sensing on the smartphone and identifies the key barriers and potential solutions of implementing opportunistic and context-aware affect sensing on smartphone platforms.