Model-based Task Analysis and Large-scale Video-based Remote Evaluation Methods for Extended Reality Research
This work addresses the problem of remote XR evaluation for researchers, but it is incremental as it adapts existing methods to a new context.
The paper tackled the challenge of conducting extended reality (XR) research during the COVID-19 pandemic by introducing two remote methods: a predictive model-based task analysis and a large-scale video-based evaluation, using a box stacking task with three interaction modalities and involving 118 participants to measure workload and usability.
In this paper, we introduce two remote extended reality (XR) research methods that can overcome the limitations of lab-based controlled experiments, especially during the COVID-19 pandemic: (1) a predictive model-based task analysis and (2) a large-scale video-based remote evaluation. We used a box stacking task including three interaction modalities - two multimodal gaze-based interactions as well as a unimodal hand-based interaction which is defined as our baseline. For the first evaluation, a GOMS-based task analysis was performed by analyzing the tasks to understand human behaviors in XR and predict task execution times. For the second evaluation, an online survey was administered using a series of the first-person point of view videos where a user performs the corresponding task with three interaction modalities. A total of 118 participants were asked to compare the interaction modes based on their judgment. Two standard questionnaires were used to measure perceived workload and the usability of the modalities.