Graph Attention Transformer Network for Multi-Label Image Classification
This work addresses the problem of improving generalization in multi-label image classification for computer vision applications, representing an incremental advance over existing methods.
The paper tackles the challenge of learning inter-label correlations for multi-label image classification by proposing a Graph Attention Transformer Network (GATN), which uses cosine similarity from label word embeddings and a graph attention transformer layer to adapt the correlation matrix, achieving state-of-the-art performance on three datasets.
Multi-label classification aims to recognize multiple objects or attributes from images. However, it is challenging to learn from proper label graphs to effectively characterize such inter-label correlations or dependencies. Current methods often use the co-occurrence probability of labels based on the training set as the adjacency matrix to model this correlation, which is greatly limited by the dataset and affects the model's generalization ability. In this paper, we propose a Graph Attention Transformer Network (GATN), a general framework for multi-label image classification that can effectively mine complex inter-label relationships. First, we use the cosine similarity based on the label word embedding as the initial correlation matrix, which can represent rich semantic information. Subsequently, we design the graph attention transformer layer to transfer this adjacency matrix to adapt to the current domain. Our extensive experiments have demonstrated that our proposed methods can achieve state-of-the-art performance on three datasets.