Accelerated Training for Massive Classification via Dynamic Class Selection
This addresses a critical bottleneck for real-world systems like face recognition that require classification over millions of classes, though it is an incremental improvement over existing methods.
The paper tackles the problem of excessive memory and computational cost in training deep networks for massive classification tasks with hundreds of thousands of classes, by introducing a method that dynamically selects active classes per mini-batch, resulting in significant reductions in training cost and memory demand while maintaining competitive performance on large-scale benchmarks.
Massive classification, a classification task defined over a vast number of classes (hundreds of thousands or even millions), has become an essential part of many real-world systems, such as face recognition. Existing methods, including the deep networks that achieved remarkable success in recent years, were mostly devised for problems with a moderate number of classes. They would meet with substantial difficulties, e.g. excessive memory demand and computational cost, when applied to massive problems. We present a new method to tackle this problem. This method can efficiently and accurately identify a small number of "active classes" for each mini-batch, based on a set of dynamic class hierarchies constructed on the fly. We also develop an adaptive allocation scheme thereon, which leads to a better tradeoff between performance and cost. On several large-scale benchmarks, our method significantly reduces the training cost and memory demand, while maintaining competitive performance.