CSG0: Continual Urban Scene Generation with Zero Forgetting
This addresses the challenge of maintaining synthesis quality across multiple domains in continual learning for applications like automated driving, though it is incremental as it builds on existing GAN and continual learning techniques.
The paper tackles the problem of catastrophic forgetting in continual training of GANs for urban scene generation, achieving zero forgetting with improved synthesis quality and outperforming brute-force methods, especially in low-data regimes.
With the rapid advances in generative adversarial networks (GANs), the visual quality of synthesised scenes keeps improving, including for complex urban scenes with applications to automated driving. We address in this work a continual scene generation setup in which GANs are trained on a stream of distinct domains; ideally, the learned models should eventually be able to generate new scenes in all seen domains. This setup reflects the real-life scenario where data are continuously acquired in different places at different times. In such a continual setup, we aim for learning with zero forgetting, \IE, with no degradation in synthesis quality over earlier domains due to catastrophic forgetting. To this end, we introduce a novel framework that not only (i) enables seamless knowledge transfer in continual training but also (ii) guarantees zero forgetting with a small overhead cost. While being more memory efficient, thanks to continual learning, our model obtains better synthesis quality as compared against the brute-force solution that trains one full model for each domain. Especially, under extreme low-data regimes, our approach outperforms the brute-force one by a large margin.