Large Language Models Preserve Semantic Isotopies in Story Continuations
This addresses the problem of semantic coherence in LLM-generated text for NLP researchers and practitioners, though it appears incremental by extending previous insights.
The study investigated whether large language models preserve semantic isotopies in story continuations, finding that LLM completion within a given token horizon preserves semantic isotopies across multiple structural and semantic properties.
In this work, we explore the relevance of textual semantics to Large Language Models (LLMs), extending previous insights into the connection between distributional semantics and structural semantics. We investigate whether LLM-generated texts preserve semantic isotopies. We design a story continuation experiment using 10,000 ROCStories prompts completed by five LLMs. We first validate GPT-4o's ability to extract isotopies from a linguistic benchmark, then apply it to the generated stories. We then analyze structural (coverage, density, spread) and semantic properties of isotopies to assess how they are affected by completion. Results show that LLM completion within a given token horizon preserves semantic isotopies across multiple properties.