Autoencoding Hyperbolic Representation for Adversarial Generation
This work addresses a crucial problem for researchers and practitioners in geometric deep learning by enabling stable hyperbolic generative models, though it is incremental as it builds on existing hyperbolic and GAN methods.
The paper tackles the problem of numerical instability in hyperbolic neural networks, which hinders building generative models for complex data, by proposing HAEGAN, a hyperbolic autoencoding GAN with novel architecture and layers that achieves state-of-the-art structure-related performance in generating complex data.
With the recent advance of geometric deep learning, neural networks have been extensively used for data in non-Euclidean domains. In particular, hyperbolic neural networks have proved successful in processing hierarchical information of data. However, many hyperbolic neural networks are numerically unstable during training, which precludes using complex architectures. This crucial problem makes it difficult to build hyperbolic generative models for real and complex data. In this work, we propose a hyperbolic generative network in which we design novel architecture and layers to improve stability in training. Our proposed network contains three parts: first, a hyperbolic autoencoder (AE) that produces hyperbolic embedding for input data; second, a hyperbolic generative adversarial network (GAN) for generating the hyperbolic latent embedding of the AE from simple noise; third, a generator that inherits the decoder from the AE and the generator from the GAN. We call this network the hyperbolic AE-GAN, or HAEGAN for short. The architecture of HAEGAN fosters expressive representation in the hyperbolic space, and the specific design of layers ensures numerical stability. Experiments show that HAEGAN is able to generate complex data with state-of-the-art structure-related performance.