FusedProp: Towards Efficient Training of Generative Adversarial Networks
This addresses efficiency issues for researchers and practitioners training GANs, though it is incremental with preliminary results requiring further stability improvements.
The paper tackles the problem of computationally expensive training in state-of-the-art generative adversarial networks (GANs) by proposing the FusedProp algorithm, which achieves 1.49 times faster training speed compared to conventional methods.
Generative adversarial networks (GANs) are capable of generating strikingly realistic samples but state-of-the-art GANs can be extremely computationally expensive to train. In this paper, we propose the fused propagation (FusedProp) algorithm which can be used to efficiently train the discriminator and the generator of common GANs simultaneously using only one forward and one backward propagation. We show that FusedProp achieves 1.49 times the training speed compared to the conventional training of GANs, although further studies are required to improve its stability. By reporting our preliminary results and open-sourcing our implementation, we hope to accelerate future research on the training of GANs.