A spin-glass model for the loss surfaces of generative adversarial networks
This research provides theoretical insights into the complex loss landscapes of GANs, which is a problem for researchers and practitioners working on training and understanding these models.
This paper introduces a new mathematical model using two interacting spin glasses to represent the key design features of Generative Adversarial Networks (GANs). Through theoretical analysis using Random Matrix Theory, the study provides insights into the complexity of critical points on the loss surfaces of large GANs, revealing both familiar and novel structures.
We present a novel mathematical model that seeks to capture the key design feature of generative adversarial networks (GANs). Our model consists of two interacting spin glasses, and we conduct an extensive theoretical analysis of the complexity of the model's critical points using techniques from Random Matrix Theory. The result is insights into the loss surfaces of large GANs that build upon prior insights for simpler networks, but also reveal new structure unique to this setting.