Explaining generative diffusion models via visual analysis for interpretable decision-making process
This work addresses the interpretability of diffusion models for researchers and practitioners, but it is incremental as it builds on existing methods with new visualization tools.
The authors tackled the problem of explaining the diffusion process in generative diffusion models, which is challenging due to the difficulty in interpreting noisy image sequences, by proposing tools for visualizing and analyzing the process to make it human-understandable, with results substantiated using metrics like AUC score, correlation quantification, and cross-attention mapping.
Diffusion models have demonstrated remarkable performance in generation tasks. Nevertheless, explaining the diffusion process remains challenging due to it being a sequence of denoising noisy images that are difficult for experts to interpret. To address this issue, we propose the three research questions to interpret the diffusion process from the perspective of the visual concepts generated by the model and the region where the model attends in each time step. We devise tools for visualizing the diffusion process and answering the aforementioned research questions to render the diffusion process human-understandable. We show how the output is progressively generated in the diffusion process by explaining the level of denoising and highlighting relationships to foundational visual concepts at each time step through the results of experiments with various visual analyses using the tools. Throughout the training of the diffusion model, the model learns diverse visual concepts corresponding to each time-step, enabling the model to predict varying levels of visual concepts at different stages. We substantiate our tools using Area Under Cover (AUC) score, correlation quantification, and cross-attention mapping. Our findings provide insights into the diffusion process and pave the way for further research into explainable diffusion mechanisms.