Objestures: Everyday Objects Meet Mid-Air Gestures for Expressive Interaction
This work addresses a design problem for developers and designers in human-computer interaction by providing a more generalizable framework for expressive interactions, though it is incremental in building on prior integration efforts.
The paper tackled the lack of generalized guidance for integrating everyday object-based and mid-air gesture interactions by introducing Objestures, a design space of five interaction types, and found in a user study (N=12) that it achieved performance comparable to native freehand manipulation in basic 3D tasks while being perceived as intuitive and engaging.
Everyday object-based interactions (EOIs) and mid-air gesture interactions (MAIs) have been widely explored, yet prior work on their integration often targets narrow use cases or specific technologies, leaving designers and developers with limited guidance that generalizes across diverse EOIs and MAIs. We introduce Objestures ("Obj" + "Gestures") -- five interaction types spanning EOIs and MAIs, forming a design space for expressive uni- and bimanual interaction. To evaluate the usefulness of Objestures, we conducted an exploratory user study (N=12) on basic 3D tasks (rotation and scaling), which showed performance comparable to the headset's native freehand manipulation. To understand the user experience, we conducted case studies with the same participants across three applications (Sound, Draw, and Shadow), where participants found the interactions intuitive, engaging, and expressive, and indicated interest in everyday use. We further demonstrate the potential of Objestures across diverse contexts through 30 examples, and discuss limitations and implications. https://www.zhuoyuelyu.com/objestures