Towards an End-to-End System for 3D Tracking of Physical Objects in Virtual Immersive Environments
For VR/XR training applications, this provides a practical, low-cost marker-based tracking solution, though it is incremental over existing fiducial marker methods.
This work presents an end-to-end system for real-time 3D tracking of physical objects in VR using fiducial markers, enabling plug-and-play object tracking without complex hardware. The system integrates AruCo, AprilTag, and an original Colored Control Points marker system, and evaluates tag sizes, detection distances, and camera devices against theoretical limits.
This work aims to establish an end-to-end system for tracking of physical 3D objects for virtual reality (VR) applications. We focus on training applications requiring real-time tracking of the position of small physical objects and their reflection in VR space. Out goal is to perform object tracking in a "plug and play" manner, without using complex systems with quite large tracking devices or manually implementing object tracking. We therefore propose a system for object tracking via fiducial markers alongside a software harness, to enable fast and efficient designation of objects to be tracked and data streaming solution for end-use applications. The system utilizes AruCo, AprilTag and an original Colored Control Points based fiducial system. It allows for easy tag detection and use of object position data, which are crucial for immersive training environments based on VR and eXtended Reality (XR). We evaluate various tag sizes, detection distances, and different camera devices against the theoretical limits. In effect, we create a complete solution for implementing marker-based, real-to-virtual object position mapping for various applications.