Perceptual Requirements for Low-Latency Head-Mounted Displays
This work addresses the challenge of understanding latency effects on user experience in HMDs, which is incremental as it builds on existing perceptual studies by enabling lower latency measurements.
The researchers tackled the problem of measuring the perceptual impact of end-to-end latency in head-mounted displays by developing a low-latency HMD called Camsicle, which achieved 2-millisecond latency and enabled user studies showing preferences for lower latencies (e.g., 2 and 14.3 ms over 23 and 29 ms) in a ball-catching task.
End-to-end (e2e) latency in head-mounted displays (HMD) is the time delay between a physical change in the world (e.g., a user's head movement) and the moment the display updates to reflect that change. Tracking, rendering, and other computation in real systems invariably introduce some amount of e2e latency to all HMDs. In modern devices this latency is usually in the range of 12-60 milliseconds which is partially addressed through pose prediction and late stage reprojection which means that perceptual studies and user experience evaluations cannot explore latencies below these values. Here, we introduce a video passthrough HMD, called Camsicle, which is capable of 2-millisecond e2e latency and, additionally, uses a catadioptric design to achieve perspective-correct passthrough without reprojection. This platform enables naturalistic user studies to interrogate the impacts of latency on user experience, preference, and performance. Across two user studies and 57 participants we find that 2 and 14.3 millisecond latencies are preferred over 23 and 29 milliseconds when attempting to catch a ball. Additionally, we compare individual latency preferences in this naturalistic ball-catching task to psychophysical thresholds for latency detection in a reference-grade system with zero latency to investigate how psychophysical thresholds may relate to subjective evaluations in naturalistic scenarios.