The Devil is in the Tails: Fine-grained Classification in the Wild
This addresses the challenge of handling many categories with few training examples in computer vision, which is incremental as it analyzes existing methods on a new dataset.
The paper tackles the problem of fine-grained classification in long-tailed distributions, finding that while peak performance on well-represented categories is excellent, classification performance decays precipitously as training examples decrease, with transfer learning being virtually absent in current methods.
The world is long-tailed. What does this mean for computer vision and visual recognition? The main two implications are (1) the number of categories we need to consider in applications can be very large, and (2) the number of training examples for most categories can be very small. Current visual recognition algorithms have achieved excellent classification accuracy. However, they require many training examples to reach peak performance, which suggests that long-tailed distributions will not be dealt with well. We analyze this question in the context of eBird, a large fine-grained classification dataset, and a state-of-the-art deep network classification algorithm. We find that (a) peak classification performance on well-represented categories is excellent, (b) given enough data, classification performance suffers only minimally from an increase in the number of classes, (c) classification performance decays precipitously as the number of training examples decreases, (d) surprisingly, transfer learning is virtually absent in current methods. Our findings suggest that our community should come to grips with the question of long tails.