TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation
This work addresses the interpretability and controllability of DiTs for image generation, enabling applications like safe image editing and style transfer, but it is incremental as it builds on existing DiT methods.
The paper tackled the problem of extracting interpretable features from Diffusion Transformers (DiTs) by proposing TIDE, a framework that uses temporal-aware sparse autoencoders to capture sparse, interpretable activation features across timesteps, revealing hierarchical semantics like 3D structure and object class during pretraining, and it enhances interpretability and controllability while maintaining reasonable generation quality.
Diffusion Transformers (DiTs) are a powerful yet underexplored class of generative models compared to U-Net-based diffusion architectures. We propose TIDE-Temporal-aware sparse autoencoders for Interpretable Diffusion transformErs-a framework designed to extract sparse, interpretable activation features across timesteps in DiTs. TIDE effectively captures temporally-varying representations and reveals that DiTs naturally learn hierarchical semantics (e.g., 3D structure, object class, and fine-grained concepts) during large-scale pretraining. Experiments show that TIDE enhances interpretability and controllability while maintaining reasonable generation quality, enabling applications such as safe image editing and style transfer.