Just-DREAM-about-it: Figurative Language Understanding with DREAM-FLUTE
This addresses the problem of figurative language comprehension for natural language processing systems, representing an incremental improvement with a specific benchmark gain.
The paper tackled the challenge of understanding figurative language by proposing DREAM-FLUTE, a system that forms a mental model of described scenes before making decisions, achieving joint first place with 63.3% accuracy in the FigLang2022 Shared Task and showing a 3.3% improvement over standard fine-tuning.
Figurative language (e.g., "he flew like the wind") is challenging to understand, as it is hard to tell what implicit information is being conveyed from the surface form alone. We hypothesize that to perform this task well, the reader needs to mentally elaborate the scene being described to identify a sensible meaning of the language. We present DREAM-FLUTE, a figurative language understanding system that does this, first forming a "mental model" of situations described in a premise and hypothesis before making an entailment/contradiction decision and generating an explanation. DREAM-FLUTE uses an existing scene elaboration model, DREAM, for constructing its "mental model." In the FigLang2022 Shared Task evaluation, DREAM-FLUTE achieved (joint) first place (Acc@60=63.3%), and can perform even better with ensemble techniques, demonstrating the effectiveness of this approach. More generally, this work suggests that adding a reflective component to pretrained language models can improve their performance beyond standard fine-tuning (3.3% improvement in Acc@60).