Salience-SGG: Enhancing Unbiased Scene Graph Generation with Iterative Salience Estimation
This addresses biased models in scene graph generation for computer vision applications, offering an incremental improvement by enhancing spatial understanding in existing unbiased methods.
The paper tackles the problem of biased scene graph generation due to long-tailed predicate distributions by introducing Salience-SGG, which uses iterative salience estimation to enhance spatial understanding, achieving state-of-the-art performance on datasets like Visual Genome and improving spatial metrics such as Pairwise Localization Average Precision.
Scene Graph Generation (SGG) suffers from a long-tailed distribution, where a few predicate classes dominate while many others are underrepresented, leading to biased models that underperform on rare relations. Unbiased-SGG methods address this issue by implementing debiasing strategies, but often at the cost of spatial understanding, resulting in an over-reliance on semantic priors. We introduce Salience-SGG, a novel framework featuring an Iterative Salience Decoder (ISD) that emphasizes triplets with salient spatial structures. To support this, we propose semantic-agnostic salience labels guiding ISD. Evaluations on Visual Genome, Open Images V6, and GQA-200 show that Salience-SGG achieves state-of-the-art performance and improves existing Unbiased-SGG methods in their spatial understanding as demonstrated by the Pairwise Localization Average Precision