Scalable Annotation of Fine-Grained Categories Without Experts
It addresses the challenge of scalable annotation for visually similar synthetic categories, which is incremental as it builds on crowdsourcing methods.
The paper tackles the problem of annotating fine-grained categories like cars without experts by introducing a crowdsourcing workflow, resulting in a dataset of 712,430 images across 2,657 categories at 1/20th the cost of expert annotation.
We present a crowdsourcing workflow to collect image annotations for visually similar synthetic categories without requiring experts. In animals, there is a direct link between taxonomy and visual similarity: e.g. a collie (type of dog) looks more similar to other collies (e.g. smooth collie) than a greyhound (another type of dog). However, in synthetic categories such as cars, objects with similar taxonomy can have very different appearance: e.g. a 2011 Ford F-150 Supercrew-HD looks the same as a 2011 Ford F-150 Supercrew-LL but very different from a 2011 Ford F-150 Supercrew-SVT. We introduce a graph based crowdsourcing algorithm to automatically group visually indistinguishable objects together. Using our workflow, we label 712,430 images by ~1,000 Amazon Mechanical Turk workers; resulting in the largest fine-grained visual dataset reported to date with 2,657 categories of cars annotated at 1/20th the cost of hiring experts.