Jialin Wang

Yiming Luo, Jialin Wang, Rongkai Shi et al.

In recent years, Virtual Reality (VR) Head-Mounted Displays (HMD) have been used to provide an immersive, first-person view in real-time for the remote-control of Unmanned Ground Vehicles (UGV). One critical issue is that it is challenging to perceive the distance of obstacles surrounding the vehicle from 2D views in the HMD, which deteriorates the control of UGV. Conventional distance indicators used in HMD take up screen space which leads clutter on the display and can further reduce situation awareness of the physical environment. To address the issue, in this paper we propose off-screen in-device feedback using vibro-tactile and/or light-visual cues to provide real-time distance information for the remote control of UGV. Results from a study show a significantly better performance with either feedback type, reduced workload and improved usability in a driving task that requires continuous perception of the distance between the UGV and its environmental objects or obstacles. Our findings show a solid case for in-device vibro-tactile and/or light-visual feedback to support remote operation of UGVs that highly relies on distance perception of objects.

Yiming Luo, Jialin Wang, Hai-Ning Liang et al.

Virtual reality (VR) head-mounted displays (HMD) have recently been used to provide an immersive, first-person vision/view in real-time for manipulating remotely-controlled unmanned ground vehicles (UGV). The teleoperation of UGV can be challenging for operators when it is done in real time. One big challenge is for operators to perceive quickly and rapidly the distance of objects that are around the UGV while it is moving. In this research, we explore the use of monoscopic and stereoscopic views and display types (immersive and non-immersive VR) for operating vehicles remotely. We conducted two user studies to explore their feasibility and advantages. Results show a significantly better performance when using an immersive display with stereoscopic view for dynamic, real-time navigation tasks that require avoiding both moving and static obstacles. The use of stereoscopic view in an immersive display in particular improved user performance and led to better usability.

Jialin Wang, Hai-Ning Liang, Diego Monteiro et al.

Virtual Reality (VR) technology has been proliferating in the last decade, especially in the last few years. However, Simulator Sickness (SS) still represents a significant problem for its wider adoption. Currently, the most common way to detect SS is using the Simulator Sickness Questionnaire (SSQ). SSQ is a subjective measurement and is inadequate for real-time applications such as VR games. This research aims to investigate how to use machine learning techniques to detect SS based on in-game characters' and users' physiological data during gameplay in VR games. To achieve this, we designed an experiment to collect such data with three types of games. We trained a Long Short-Term Memory neural network with the dataset eye-tracking and character movement data to detect SS in real-time. Our results indicate that, in VR games, our model is an accurate and efficient way to detect SS in real-time.

Diego Monteiro, Hai-Ning Liang, Jialin Wang et al.

The amount of interest in Virtual Reality (VR) research has significantly increased over the past few years, both in academia and industry. The release of commercial VR Head-Mounted Displays (HMDs) has been a major contributing factor. However, there is still much to be learned, especially how views and input techniques, as well as their interaction, affect the VR experience. There is little work done on First-Person Shooter (FPS) games in VR, and those few studies have focused on a single aspect of VR FPS. They either focused on the view, e.g., comparing VR to a typical 2D display or on the controller types. To the best of our knowledge, there are no studies investigating variations of 2D/3D views in HMDs, controller types, and their interactions. As such, it is challenging to distinguish findings related to the controller type from those related to the view. If a study does not control for the input method and finds that 2D displays lead to higher performance than VR, we cannot generalize the results because of the confounding variables. To understand their interaction, we propose to analyze in more depth, whether it is the view (2D vs. 3D) or the way it is controlled that gives the platforms their respective advantages. To study the effects of the 2D/3D views, we created a 2D visual technique, PlaneFrame, that was applied inside the VR headset. Our results show that the controller type can have a significant positive impact on performance, immersion, and simulator sickness when associated with a 2D view. They further our understanding of the interactions that controllers and views have and demonstrate that comparisons are highly dependent on how both factors go together. Further, through a series of three experiments, we developed a technique that can lead to a substantial performance, a good level of immersion, and can minimize the level of simulator sickness.