Adversarially robust segmentation models learn perceptually-aligned gradients
This work addresses the interpretability of neural networks for researchers in computer vision, though it appears incremental as it builds on prior findings about adversarial training in classifiers.
The paper tackles the problem of leveraging adversarially-trained semantic segmentation networks for image inpainting and generation, demonstrating that these networks are more robust and exhibit perceptually-aligned gradients that produce plausible results.
The effects of adversarial training on semantic segmentation networks has not been thoroughly explored. While previous work has shown that adversarially-trained image classifiers can be used to perform image synthesis, we have yet to understand how best to leverage an adversarially-trained segmentation network to do the same. Using a simple optimizer, we demonstrate that adversarially-trained semantic segmentation networks can be used to perform image inpainting and generation. Our experiments demonstrate that adversarially-trained segmentation networks are more robust and indeed exhibit perceptually-aligned gradients which help in producing plausible image inpaintings. We seek to place additional weight behind the hypothesis that adversarially robust models exhibit gradients that are more perceptually-aligned with human vision. Through image synthesis, we argue that perceptually-aligned gradients promote a better understanding of a neural network's learned representations and aid in making neural networks more interpretable.