Generative Myopia: Why Diffusion Models Fail at Structure
This addresses a critical failure mode in generative models for combinatorial tasks like graph sparsification, which is incremental as it builds on existing diffusion methods with a novel spectral prior.
The paper tackles the failure of Graph Diffusion Models (GDMs) in preserving structurally critical but statistically rare edges, such as 'rare bridges' in graph sparsification, by identifying a phenomenon called Generative Myopia driven by Gradient Starvation. It introduces Spectrally-Weighted Diffusion, which uses Effective Resistance to re-align the objective, achieving 100% connectivity on adversarial benchmarks where standard diffusion fails completely (0%).
Graph Diffusion Models (GDMs) optimize for statistical likelihood, implicitly acting as \textbf{frequency filters} that favor abundant substructures over spectrally critical ones. We term this phenomenon \textbf{Generative Myopia}. In combinatorial tasks like graph sparsification, this leads to the catastrophic removal of ``rare bridges,'' edges that are structurally mandatory ($R_{\text{eff}} \approx 1$) but statistically scarce. We prove theoretically and empirically that this failure is driven by \textbf{Gradient Starvation}: the optimization landscape itself suppresses rare structural signals, rendering them unlearnable regardless of model capacity. To resolve this, we introduce \textbf{Spectrally-Weighted Diffusion}, which re-aligns the variational objective using Effective Resistance. We demonstrate that spectral priors can be amortized into the training phase with zero inference overhead. Our method eliminates myopia, matching the performance of an optimal Spectral Oracle and achieving \textbf{100\% connectivity} on adversarial benchmarks where standard diffusion fails completely (0\%).