Dynamic Object Comprehension: A Framework For Evaluating Artificial Visual Perception
This work targets developers and researchers in AR/MR by highlighting an incremental need for better evaluation methods to bridge the gap between existing computer vision techniques and application requirements.
The paper addresses the lack of suitable evaluation metrics for visual perception in augmented and mixed reality, proposing new criteria to assess progress in this area.
Augmented and Mixed Reality are emerging as likely successors to the mobile internet. However, many technical challenges remain. One of the key requirements of these systems is the ability to create a continuity between physical and virtual worlds, with the user's visual perception as the primary interface medium. Building this continuity requires the system to develop a visual understanding of the physical world. While there has been significant recent progress in computer vision and AI techniques such as image classification and object detection, success in these areas has not yet led to the visual perception required for these critical MR and AR applications. A significant issue is that current evaluation criteria are insufficient for these applications. To motivate and evaluate progress in this emerging area, there is a need for new metrics. In this paper we outline limitations of current evaluation criteria and propose new criteria.