Same Image, Different Meanings: Toward Retrieval of Context-Dependent Meanings
For information retrieval researchers, this work identifies and formalizes the open problem of context-dependent image retrieval in narrative settings.
The paper investigates how image meaning depends on narrative context, proposing an L1-L4 framework to organize semantics from context-independent to maximally context-dependent. Experiments show that concrete queries are retrievable without context, abstract levels require narrative grounding, and the most abstract level remains challenging even with full context.
A scene of two people in the rain can convey hope and warmth in a reunion story or sorrow and finality in a farewell story. We investigate this context-dependent nature of image meaning and its implications for retrieval. Our key observation is that context dependency correlates with semantic abstraction: concrete elements (objects, actions) remain stable across contexts, while abstract elements (atmosphere, intent) shift with context. We operationalize this as the L1--L4 framework, organizing image semantics from context-independent (L1) to maximally context-dependent (L4). Using synthetic story contexts and queries for controlled evaluation, we examine how injecting narrative context into embeddings affects retrieval across abstraction levels. Concrete queries are retrievable without context, while abstract levels increasingly depend on narrative grounding. Where context is injected also matters, with image-side enrichment proving particularly effective. The most abstract level, however, remains challenging even with full context, highlighting context-dependent image retrieval as an important open problem. Our framework and findings lay groundwork toward retrieval systems that handle the context-dependent meanings images acquire in narrative settings.