Order-Free RNN with Visual Attention for Multi-Label Classification
This work addresses multi-label classification for computer vision applications, offering an incremental improvement by combining existing attention and LSTM techniques in a novel way.
The paper tackles the problem of multi-label classification by proposing a model that integrates visual attention and LSTM without requiring pre-defined label sequences, achieving robust inference that prevents error propagation and efficiently predicts multiple labels using beam search.
In this paper, we propose the joint learning attention and recurrent neural network (RNN) models for multi-label classification. While approaches based on the use of either model exist (e.g., for the task of image captioning), training such existing network architectures typically require pre-defined label sequences. For multi-label classification, it would be desirable to have a robust inference process, so that the prediction error would not propagate and thus affect the performance. Our proposed model uniquely integrates attention and Long Short Term Memory (LSTM) models, which not only addresses the above problem but also allows one to identify visual objects of interests with varying sizes without the prior knowledge of particular label ordering. More importantly, label co-occurrence information can be jointly exploited by our LSTM model. Finally, by advancing the technique of beam search, prediction of multiple labels can be efficiently achieved by our proposed network model.