Learning to Generate Compositional Color Descriptions
This work addresses the challenge of grounded language generation for color descriptions, which is incremental as it builds on existing methods to handle compositional complexity in a specific domain.
The paper tackled the problem of generating compositional color descriptions, which are vague and complex, by using recurrent neural networks with a Fourier-transformed color representation, resulting in a model that outperforms previous work on a conditional language modeling task and accurately produces diverse descriptors like 'greenish' and 'faded teal' not seen in training.
The production of color language is essential for grounded language generation. Color descriptions have many challenging properties: they can be vague, compositionally complex, and denotationally rich. We present an effective approach to generating color descriptions using recurrent neural networks and a Fourier-transformed color representation. Our model outperforms previous work on a conditional language modeling task over a large corpus of naturalistic color descriptions. In addition, probing the model's output reveals that it can accurately produce not only basic color terms but also descriptors with non-convex denotations ("greenish"), bare modifiers ("bright", "dull"), and compositional phrases ("faded teal") not seen in training.