Discriminator-Cooperated Feature Map Distillation for GAN Compression
This work addresses the high computational demands of GANs for image generation, offering a novel compression technique that improves performance metrics, though it is incremental in the context of existing GAN compression methods.
The paper tackles the problem of compressing Generative Adversarial Networks (GANs) to reduce storage and computation costs by proposing a discriminator-cooperated distillation method, which achieves a lower FID metric of 48.24 compared to 51.92 with state-of-the-art methods after reducing over 40x MACs and 80x parameters of CycleGAN.
Despite excellent performance in image generation, Generative Adversarial Networks (GANs) are notorious for its requirements of enormous storage and intensive computation. As an awesome ''performance maker'', knowledge distillation is demonstrated to be particularly efficacious in exploring low-priced GANs. In this paper, we investigate the irreplaceability of teacher discriminator and present an inventive discriminator-cooperated distillation, abbreviated as DCD, towards refining better feature maps from the generator. In contrast to conventional pixel-to-pixel match methods in feature map distillation, our DCD utilizes teacher discriminator as a transformation to drive intermediate results of the student generator to be perceptually close to corresponding outputs of the teacher generator. Furthermore, in order to mitigate mode collapse in GAN compression, we construct a collaborative adversarial training paradigm where the teacher discriminator is from scratch established to co-train with student generator in company with our DCD. Our DCD shows superior results compared with existing GAN compression methods. For instance, after reducing over 40x MACs and 80x parameters of CycleGAN, we well decrease FID metric from 61.53 to 48.24 while the current SoTA method merely has 51.92. This work's source code has been made accessible at https://github.com/poopit/DCD-official.