Towards Interaction Around Unmodified Camera-equipped Mobile Devices
This addresses the need for extended input capabilities on mobile devices without hardware modifications, though it is incremental as it builds on existing around-device interaction concepts.
The paper tackled the problem of limited input space on mobile devices by enabling around-device interaction using built-in front-facing cameras, achieving an input reach comparable to large touch displays through hand pose estimation via reflections in eyewear.
Around-device interaction promises to extend the input space of mobile and wearable devices beyond the common but restricted touchscreen. So far, most around-device interaction approaches rely on instrumenting the device or the environment with additional sensors. We believe, that the full potential of ordinary cameras, specifically user-facing cameras, which are integrated in most mobile devices today, are not used to their full potential, yet. We To this end, we present a novel approach for extending the input space around unmodified mobile devices using built-in front-facing cameras of unmodified handheld devices. Our approach estimates hand poses and gestures through reflections in sunglasses, ski goggles or visors. Thereby, GlassHands creates an enlarged input space, rivaling input reach on large touch displays. We discuss the idea, its limitations and future work.