Predicate Classification Using Optimal Transport Loss in Scene Graph Generation
This work addresses a specific bottleneck in scene graph generation for computer vision applications, offering an incremental improvement over existing methods.
The paper tackles the problem of biased predictions in scene graph generation due to imbalanced relationship labels by proposing a method that uses optimal transport loss for predicate classification, resulting in improved mean Recall@50 and 100 and better recall for rare labels.
In scene graph generation (SGG), learning with cross-entropy loss yields biased predictions owing to the severe imbalance in the distribution of the relationship labels in the dataset. Thus, this study proposes a method to generate scene graphs using optimal transport as a measure for comparing two probability distributions. We apply learning with the optimal transport loss, which reflects the similarity between the labels in terms of transportation cost, for predicate classification in SGG. In the proposed approach, the transportation cost of the optimal transport is defined using the similarity of words obtained from the pre-trained model. The experimental evaluation of the effectiveness demonstrates that the proposed method outperforms existing methods in terms of mean Recall@50 and 100. Furthermore, it improves the recall of the relationship labels scarcely available in the dataset.