CAGAN: Text-To-Image Generation with Combined Attention GANs
It addresses image generation from text for AI applications, but is incremental with hybrid attention methods.
The paper tackles text-to-image generation by proposing CAGAN, which combines word and squeeze-and-excitation attention to address channel relationships, improving state-of-the-art IS and FID scores on CUB and COCO datasets, such as enhancing FID on COCO.
Generating images according to natural language descriptions is a challenging task. Prior research has mainly focused to enhance the quality of generation by investigating the use of spatial attention and/or textual attention thereby neglecting the relationship between channels. In this work, we propose the Combined Attention Generative Adversarial Network (CAGAN) to generate photo-realistic images according to textual descriptions. The proposed CAGAN utilises two attention models: word attention to draw different sub-regions conditioned on related words; and squeeze-and-excitation attention to capture non-linear interaction among channels. With spectral normalisation to stabilise training, our proposed CAGAN improves the state of the art on the IS and FID on the CUB dataset and the FID on the more challenging COCO dataset. Furthermore, we demonstrate that judging a model by a single evaluation metric can be misleading by developing an additional model adding local self-attention which scores a higher IS, outperforming the state of the art on the CUB dataset, but generates unrealistic images through feature repetition.