Unbiased Scene Graph Generation by Type-Aware Message Passing on Heterogeneous and Dual Graphs
This work addresses the challenge of improving predictive performance for tail classes in scene graph generation, which is important for computer vision applications, but it appears incremental as it builds on existing unbiased methods.
The paper tackles the problem of unbiased scene graph generation by addressing performance issues for both head and tail classes, proposing a method that uses heterogeneous and dual graphs with type-aware message passing, resulting in improvements in R@K and mR@K metrics on two datasets.
Although great progress has been made in the research of unbiased scene graph generation, issues still hinder improving the predictive performance of both head and tail classes. An unbiased scene graph generation (TA-HDG) is proposed to address these issues. For modeling interactive and non-interactive relations, the Interactive Graph Construction is proposed to model the dependence of relations on objects by combining heterogeneous and dual graph, when modeling relations between multiple objects. It also implements a subject-object pair selection strategy to reduce meaningless edges. Moreover, the Type-Aware Message Passing enhances the understanding of complex interactions by capturing intra- and inter-type context in the Intra-Type and Inter-Type stages. The Intra-Type stage captures the semantic context of inter-relaitons and inter-objects. On this basis, the Inter-Type stage captures the context between objects and relations for interactive and non-interactive relations, respectively. Experiments on two datasets show that TA-HDG achieves improvements in the metrics of R@K and mR@K, which proves that TA-HDG can accurately predict the tail class while maintaining the competitive performance of the head class.