Conditional Hybrid GAN for Sequence Generation
This addresses a specific bottleneck in generative models for multi-attribute sequence generation, such as in music composition, but is incremental as it builds on existing GAN techniques.
The paper tackles the problem of generating sequences with multiple attributes conditioned on context, proposing a conditional hybrid GAN that uses relational reasoning and Gumbel-Softmax to handle discrete data, and demonstrates improved performance in generating melodies from lyrics compared to existing methods.
Conditional sequence generation aims to instruct the generation procedure by conditioning the model with additional context information, which is a self-supervised learning issue (a form of unsupervised learning with supervision information from data itself). Unfortunately, the current state-of-the-art generative models have limitations in sequence generation with multiple attributes. In this paper, we propose a novel conditional hybrid GAN (C-Hybrid-GAN) to solve this issue. Discrete sequence with triplet attributes are separately generated when conditioned on the same context. Most importantly, relational reasoning technique is exploited to model not only the dependency inside each sequence of the attribute during the training of the generator but also the consistency among the sequences of attributes during the training of the discriminator. To avoid the non-differentiability problem in GANs encountered during discrete data generation, we exploit the Gumbel-Softmax technique to approximate the distribution of discrete-valued sequences.Through evaluating the task of generating melody (associated with note, duration, and rest) from lyrics, we demonstrate that the proposed C-Hybrid-GAN outperforms the existing methods in context-conditioned discrete-valued sequence generation.