Aggregated Learning: A Vector-Quantization Approach to Learning Neural Network Classifiers
This work addresses classification efficiency for neural network users by introducing a novel framework that may improve learning, though it appears incremental as it builds on existing information bottleneck and quantization theories.
The paper tackles the problem of learning neural network classifiers by linking it to representation learning under the information bottleneck principle, showing equivalence to a quantization problem and proposing a vector quantization approach called Aggregated Learning, where multiple objects are jointly classified by a single network, with effectiveness verified through experiments on image recognition and text classification tasks.
We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call "IB learning". We show that IB learning is, in fact, equivalent to a special class of the quantization problem. The classical results in rate-distortion theory then suggest that IB learning can benefit from a "vector quantization" approach, namely, simultaneously learning the representations of multiple input objects. Such an approach assisted with some variational techniques, result in a novel learning framework, "Aggregated Learning", for classification with neural network models. In this framework, several objects are jointly classified by a single neural network. The effectiveness of this framework is verified through extensive experiments on standard image recognition and text classification tasks.