Ground-truth or DAER: Selective Re-query of Secondary Information
This addresses a practical issue in computer vision for applications like crowdsourcing or automated systems, though it is incremental as it builds on existing seeded tasks.
The paper tackles the problem of seed rejection in vision tasks, where noisy secondary information can degrade model performance, and proposes a training method and evaluation metrics that reduce the number of seeds needing review by over 23% compared to baselines.
Many vision tasks use secondary information at inference time -- a seed -- to assist a computer vision model in solving a problem. For example, an initial bounding box is needed to initialize visual object tracking. To date, all such work makes the assumption that the seed is a good one. However, in practice, from crowdsourcing to noisy automated seeds, this is often not the case. We hence propose the problem of seed rejection -- determining whether to reject a seed based on the expected performance degradation when it is provided in place of a gold-standard seed. We provide a formal definition to this problem, and focus on two meaningful subgoals: understanding causes of error and understanding the model's response to noisy seeds conditioned on the primary input. With these goals in mind, we propose a novel training method and evaluation metrics for the seed rejection problem. We then use seeded versions of the viewpoint estimation and fine-grained classification tasks to evaluate these contributions. In these experiments, we show our method can reduce the number of seeds that need to be reviewed for a target performance by over 23% compared to strong baselines.