What the DAAM: Interpreting Stable Diffusion Using Cross Attention
This provides interpretability for text-to-image diffusion models, enabling researchers to analyze and improve model behavior, though it is incremental as it applies existing attribution techniques to a new model type.
The authors tackled the lack of interpretability in large-scale diffusion models like Stable Diffusion by developing DAAM, a method that produces pixel-level attribution maps using cross-attention scores, and found that it achieves 0.75 F1 score for noun segmentation and identifies issues like cohyponyms worsening generation quality.
Large-scale diffusion neural networks represent a substantial milestone in text-to-image generation, but they remain poorly understood, lacking interpretability analyses. In this paper, we perform a text-image attribution analysis on Stable Diffusion, a recently open-sourced model. To produce pixel-level attribution maps, we upscale and aggregate cross-attention word-pixel scores in the denoising subnetwork, naming our method DAAM. We evaluate its correctness by testing its semantic segmentation ability on nouns, as well as its generalized attribution quality on all parts of speech, rated by humans. We then apply DAAM to study the role of syntax in the pixel space, characterizing head--dependent heat map interaction patterns for ten common dependency relations. Finally, we study several semantic phenomena using DAAM, with a focus on feature entanglement, where we find that cohyponyms worsen generation quality and descriptive adjectives attend too broadly. To our knowledge, we are the first to interpret large diffusion models from a visuolinguistic perspective, which enables future lines of research. Our code is at https://github.com/castorini/daam.