Theory of Generative Deep Learning : Probe Landscape of Empirical Error via Norm Based Capacity Control
This work addresses the problem of low interpretability in generative models for researchers, though it appears incremental as it extends existing theoretical frameworks from discriminative to generative deep learning.
The paper tackles the lack of theoretical foundation in generative deep learning by analyzing the empirical error landscape for image super-resolution using norm-based capacity control, achieving theoretical advances in interpreting training dynamics from mathematical and biological perspectives.
Despite its remarkable empirical success as a highly competitive branch of artificial intelligence, deep learning is often blamed for its widely known low interpretation and lack of firm and rigorous mathematical foundation. However, most theoretical endeavor is devoted in discriminative deep learning case, whose complementary part is generative deep learning. To the best of our knowledge, we firstly highlight landscape of empirical error in generative case to complete the full picture through exquisite design of image super resolution under norm based capacity control. Our theoretical advance in interpretation of the training dynamic is achieved from both mathematical and biological sides.