Inverting Adversarially Robust Networks for Image Synthesis
This work addresses efficiency and performance issues in image synthesis for applications like style transfer and denoising, though it is incremental in leveraging existing adversarial robustness concepts.
The paper tackles the high computational cost and complexity of training inverters for image synthesis by using adversarially robust representations as a perceptual primitive, achieving superior reconstruction quality and generalization with significantly less complexity compared to recent ImageNet feature inversion methods.
Despite unconditional feature inversion being the foundation of many image synthesis applications, training an inverter demands a high computational budget, large decoding capacity and imposing conditions such as autoregressive priors. To address these limitations, we propose the use of adversarially robust representations as a perceptual primitive for feature inversion. We train an adversarially robust encoder to extract disentangled and perceptually-aligned image representations, making them easily invertible. By training a simple generator with the mirror architecture of the encoder, we achieve superior reconstruction quality and generalization over standard models. Based on this, we propose an adversarially robust autoencoder and demonstrate its improved performance on style transfer, image denoising and anomaly detection tasks. Compared to recent ImageNet feature inversion methods, our model attains improved performance with significantly less complexity.