The Overlooked Classifier in Human-Object Interaction Recognition
This work addresses a domain-specific problem in computer vision for HOI recognition, offering a simpler and more effective method compared to existing approaches.
The paper tackled the challenges of class imbalance and multi-label learning in Human-Object Interaction (HOI) recognition by improving the classifier with language embeddings and a new loss function, achieving state-of-the-art performance without object detection or human pose data.
Human-Object Interaction (HOI) recognition is challenging due to two factors: (1) significant imbalance across classes and (2) requiring multiple labels per image. This paper shows that these two challenges can be effectively addressed by improving the classifier with the backbone architecture untouched. Firstly, we encode the semantic correlation among classes into the classification head by initializing the weights with language embeddings of HOIs. As a result, the performance is boosted significantly, especially for the few-shot subset. Secondly, we propose a new loss named LSE-Sign to enhance multi-label learning on a long-tailed dataset. Our simple yet effective method enables detection-free HOI classification, outperforming the state-of-the-arts that require object detection and human pose by a clear margin. Moreover, we transfer the classification model to instance-level HOI detection by connecting it with an off-the-shelf object detector. We achieve state-of-the-art without additional fine-tuning.