Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning
This addresses the challenge of dynamic content placement in MR for users, though it appears incremental as an initial exploration of RL for this application.
The paper tackles the problem of adaptive 3D UI placement in mixed reality by using deep reinforcement learning to position content based on users' poses and environments, with preliminary results showing potential to maximize user reward.
Mixed Reality (MR) could assist users' tasks by continuously integrating virtual content with their view of the physical environment. However, where and how to place these content to best support the users has been a challenging problem due to the dynamic nature of MR experiences. In contrast to prior work that investigates optimization-based methods, we are exploring how reinforcement learning (RL) could assist with continuous 3D content placement that is aware of users' poses and their surrounding environments. Through an initial exploration and preliminary evaluation, our results demonstrate the potential of RL to position content that maximizes the reward for users on the go. We further identify future directions for research that could harness the power of RL for personalized and optimized UI and content placement in MR.