Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?
This work addresses the gap in understanding how perceptual and discourse contexts affect information distribution in language, providing insights for computational linguistics and multimodal AI, though it is incremental as it extends prior text-only studies to grounded settings.
The study tackled the problem of whether the Uniform Information Density (UID) hypothesis holds in visually grounded language settings, finding that grounding on perception consistently increases information uniformity across 30 languages compared to text-only settings, with the strongest surprisal reductions at discourse unit onsets in visual narratives.
The Uniform Information Density (UID) hypothesis posits that speakers are subject to a communicative pressure to distribute information evenly within utterances, minimising surprisal variance. While this hypothesis has been tested empirically, prior studies are limited exclusively to text-only inputs, abstracting away from the perceptual context in which utterances are produced. In this work, we present the first computational study of UID in visually grounded settings. We estimate surprisal using multilingual vision-and-language models over image-caption data in 30 languages and visual storytelling data in 13 languages, together spanning 11 families. We find that grounding on perception consistently smooths the distribution of information, increasing both global and local uniformity across typologically diverse languages compared to text-only settings. In visual narratives, grounding in both image and discourse contexts has additional effects, with the strongest surprisal reductions occurring at the onset of discourse units. Overall, this study takes a first step towards modelling the temporal dynamics of information flow in ecologically plausible, multimodal language use, and finds that grounded language exhibits greater information uniformity, supporting a context-sensitive formulation of UID.