Combining Gesture and Voice Control for Mid-Air Manipulation of CAD Models in VR Environments
This work aims to simplify the steep learning curve and time-consuming nature of CAD modeling for designers, offering an incremental improvement in user interaction.
This paper addresses the complexity of 3D object modeling in CAD by developing a VR system that combines hand gesture recognition with voice commands. The system allows users to manipulate 3D models using a Constructive Solid Geometry (CSG) tree representation, aiming to simplify the design process.
Modeling 3D objects in domains like Computer Aided Design (CAD) is time-consuming and comes with a steep learning curve needed to master the design process as well as tool complexities. In order to simplify the modeling process, we designed and implemented a prototypical system that leverages the strengths of Virtual Reality (VR) hand gesture recognition in combination with the expressiveness of a voice-based interface for the task of 3D modeling. Furthermore, we use the Constructive Solid Geometry (CSG) tree representation for 3D models within the VR environment to let the user manipulate objects from the ground up, giving an intuitive understanding of how the underlying basic shapes connect. The system uses standard mid-air 3D object manipulation techniques and adds a set of voice commands to help mitigate the deficiencies of current hand gesture recognition techniques. A user study was conducted to evaluate the proposed prototype. The combination of our hybrid input paradigm shows to be a promising step towards easier to use CAD modeling.