ConMeC: A Dataset for Metonymy Resolution with Common Nouns
This addresses a gap in NLP for handling metonymy in common nouns, which is incremental as it extends existing work from named entities to a broader linguistic phenomenon.
The paper tackles the problem of metonymy resolution for common nouns, which is understudied compared to named entities, by creating a new dataset (ConMeC) with 6,000 annotated sentences and evaluating methods including a chain-of-thought prompting approach with LLMs, showing that LLMs achieve performance comparable to supervised BERT models on well-defined categories but struggle with nuanced cases.
Metonymy plays an important role in our daily communication. People naturally think about things using their most salient properties or commonly related concepts. For example, by saying "The bus decided to skip our stop today," we actually mean that the bus driver made the decision, not the bus. Prior work on metonymy resolution has mainly focused on named entities. However, metonymy involving common nouns (such as desk, baby, and school) is also a frequent and challenging phenomenon. We argue that NLP systems should be capable of identifying the metonymic use of common nouns in context. We create a new metonymy dataset ConMeC, which consists of 6,000 sentences, where each sentence is paired with a target common noun and annotated by humans to indicate whether that common noun is used metonymically or not in that context. We also introduce a chain-of-thought based prompting method for detecting metonymy using large language models (LLMs). We evaluate our LLM-based pipeline, as well as a supervised BERT model on our dataset and three other metonymy datasets. Our experimental results demonstrate that LLMs could achieve performance comparable to the supervised BERT model on well-defined metonymy categories, while still struggling with instances requiring nuanced semantic understanding. Our dataset is publicly available at: https://github.com/SaptGhosh/ConMeC.