Annotating References to Mythological Entities in French Literature
This work addresses a domain-specific task in digital humanities, offering a tool for automating annotations in literary analysis, though it is incremental in applying existing LLMs to a new dataset.
The paper tackles the problem of annotating references to Roman and Greek mythological entities in French literature using large language models (LLMs), finding that LLMs can perform annotations with high accuracy but struggle with information retrieval and hallucination errors.
In this paper, we explore the relevance of large language models (LLMs) for annotating references to Roman and Greek mythological entities in modern and contemporary French literature. We present an annotation scheme and demonstrate that recent LLMs can be directly applied to follow this scheme effectively, although not without occasionally making significant analytical errors. Additionally, we show that LLMs (and, more specifically, ChatGPT) are capable of offering interpretative insights into the use of mythological references by literary authors. However, we also find that LLMs struggle to accurately identify relevant passages in novels (when used as an information retrieval engine), often hallucinating and generating fabricated examples-an issue that raises significant ethical concerns. Nonetheless, when used carefully, LLMs remain valuable tools for performing annotations with high accuracy, especially for tasks that would be difficult to annotate comprehensively on a large scale through manual methods alone.