Dependency-Aware Discrete Diffusion for Scene Graph Generation
For researchers in scene graph generation and text-to-image synthesis, this work provides a method to generate structured scene graphs from text, improving compositional fidelity in image generation.
The paper introduces a dependency-aware discrete diffusion model for generating scene graphs from natural language, addressing the challenge of hierarchical dependencies in structured graphs. The method improves over baselines on graph and layout metrics and enhances compositional alignment in downstream image generation, especially in multi-object scenarios.
Scene graphs (SGs) represent objects and their relationships as structured graphs, enabling applications in image generation, robotics, and 3D understanding. Recent work suggests that conditioning image generation on scene graphs improves compositional fidelity compared to text-only prompting. However, since users typically provide text rather than structured graphs, a key challenge is to generate scene graphs from natural language. Prior work on discrete diffusion has demonstrated success in generating generic graphs such as molecules and circuits, but fails to account for the hierarchical structure and strong dependencies between objects, edges, and relations in scene graphs. We address this limitation by introducing a dependency-aware, hierarchically constrained discrete diffusion model for scene graph generation. Our approach decouples structure and semantics across the forward and reverse processes, enabling the model to capture conditional dependencies. At inference time, we perform training-free conditioning to sample text-aligned scene graphs. We evaluate our method on standard SG benchmarks and demonstrate improvements over both continuous and discrete graph generation baselines across graph and layout metrics. When fed to downstream image generation, our approach yields improved compositional alignment compared to text-to-image models, particularly in multi-object scenarios.