Generalized Unbiased Scene Graph Generation
This addresses a pervasive and challenging issue in SGG for computer vision applications, offering a more comprehensive solution beyond existing methods, though it builds incrementally on prior work focused only on predicate-level imbalance.
The paper tackles the problem of concept-level imbalance in Scene Graph Generation (SGG), which persists even when predicates are balanced, by introducing Generalized Unbiased SGG (G-USGG) and proposing the Multi-Concept Learning (MCL) framework. The result is a model-agnostic strategy that achieves new state-of-the-art performance on VG-SGG and OI-SGG datasets, improving both predicate-level unbiased relation recognition and concept-level compositional generability.
Existing Unbiased Scene Graph Generation (USGG) methods only focus on addressing the predicate-level imbalance that high-frequency classes dominate predictions of rare ones, while overlooking the concept-level imbalance. Actually, even if predicates themselves are balanced, there is still a significant concept-imbalance within them due to the long-tailed distribution of contexts (i.e., subject-object combinations). This concept-level imbalance poses a more pervasive and challenging issue compared to the predicate-level imbalance since subject-object pairs are inherently complex in combinations. Hence, we introduce a novel research problem: Generalized Unbiased Scene Graph Generation (G-USGG), which takes into account both predicate-level and concept-level imbalance. To the end, we propose the Multi-Concept Learning (MCL) framework, which ensures a balanced learning process across rare/ uncommon/ common concepts. MCL first quantifies the concept-level imbalance across predicates in terms of different amounts of concepts, representing as multiple concept-prototypes within the same class. It then effectively learns concept-prototypes by applying the Concept Regularization (CR) technique. Furthermore, to achieve balanced learning over different concepts, we introduce the Balanced Prototypical Memory (BPM), which guides SGG models to generate balanced representations for concept-prototypes. Extensive experiments demonstrate the remarkable efficacy of our model-agnostic strategy in enhancing the performance of benchmark models on both VG-SGG and OI-SGG datasets, leading to new state-of-the-art achievements in two key aspects: predicate-level unbiased relation recognition and concept-level compositional generability.