Mining Deep And-Or Object Structures via Cost-Sensitive Question-Answer-Based Active Annotations
This addresses the challenge of reducing annotation costs for detailed object structure modeling in computer vision, though it is incremental as it builds on existing active learning and AOG methods.
The paper tackled the problem of learning a compositional And-Or graph (AOG) from web images with minimal human supervision, using a cost-sensitive active Question-Answering framework, and achieved better performance with much less labeling (e.g., 3-10 training objects per category) compared to baselines.
This paper presents a cost-sensitive active Question-Answering (QA) framework for learning a nine-layer And-Or graph (AOG) from web images. The AOG explicitly represents object categories, poses/viewpoints, parts, and detailed structures within the parts in a compositional hierarchy. The QA framework is designed to minimize an overall risk, which trades off the loss and query costs. The loss is defined for nodes in all layers of the AOG, including the generative loss (measuring the likelihood of the images) and the discriminative loss (measuring the fitness to human answers). The cost comprises both the human labor of answering questions and the computational cost of model learning. The cost-sensitive QA framework iteratively selects different storylines of questions to update different nodes in the AOG. Experiments showed that our method required much less human supervision (e.g., labeling parts on 3--10 training objects for each category) and achieved better performance than baseline methods.