Topic Segmentation Using Generative Language Models
This addresses topic segmentation for text analysis, but it is incremental as it builds on prior methods with LLMs.
The paper tackles topic segmentation by using generative Large Language Models (LLMs) with an overlapping and recursive prompting strategy, showing that LLMs can be more effective than existing methods, though issues remain.
Topic segmentation using generative Large Language Models (LLMs) remains relatively unexplored. Previous methods use semantic similarity between sentences, but such models lack the long range dependencies and vast knowledge found in LLMs. In this work, we propose an overlapping and recursive prompting strategy using sentence enumeration. We also support the adoption of the boundary similarity evaluation metric. Results show that LLMs can be more effective segmenters than existing methods, but issues remain to be solved before they can be relied upon for topic segmentation.