FeatInv: Spatially resolved mapping from feature space to input space using conditional diffusion models
This provides a method for improving feature space understanding in computer vision models, though it appears incremental as it builds on existing diffusion model approaches.
The paper tackles the problem of interpreting deep neural networks' internal representations by proposing FeatInv, which uses conditional diffusion models to map from feature space to input space with high-fidelity reconstructions across CNNs and ViTs.
Internal representations are crucial for understanding deep neural networks, such as their properties and reasoning patterns, but remain difficult to interpret. While mapping from feature space to input space aids in interpreting the former, existing approaches often rely on crude approximations. We propose using a conditional diffusion model - a pretrained high-fidelity diffusion model conditioned on spatially resolved feature maps - to learn such a mapping in a probabilistic manner. We demonstrate the feasibility of this approach across various pretrained image classifiers from CNNs to ViTs, showing excellent reconstruction capabilities. Through qualitative comparisons and robustness analysis, we validate our method and showcase possible applications, such as the visualization of concept steering in input space or investigations of the composite nature of the feature space. This approach has broad potential for improving feature space understanding in computer vision models.