Dealing with a large number of classes -- Likelihood, Discrimination or Ranking?
This addresses the challenge of scalability in classification for applications with many classes, but it is incremental as it builds on existing approximations.
The paper tackles the problem of training probabilistic classifiers with a large number of classes by proposing a simple approach that approximates the likelihood, showing it works well on toy problems and is competitive with recent non-likelihood based approximations.
We consider training probabilistic classifiers in the case of a large number of classes. The number of classes is assumed too large to perform exact normalisation over all classes. To account for this we consider a simple approach that directly approximates the likelihood. We show that this simple approach works well on toy problems and is competitive with recently introduced alternative non-likelihood based approximations. Furthermore, we relate this approach to a simple ranking objective. This leads us to suggest a specific setting for the optimal threshold in the ranking objective.