Beyond Surface Similarity: Detecting Subtle Semantic Shifts in Financial Narratives
This addresses a domain-specific problem for financial market stakeholders by enabling better tracking of company changes over time, though it appears incremental as it adapts existing techniques to a new financial task.
The paper tackles the problem of detecting subtle semantic shifts in financial narratives from company statements over time, proposing a specialized LLM-augmented pipeline that outperforms existing methods on a human-annotated dataset.
In this paper, we introduce the Financial-STS task, a financial domain-specific NLP task designed to measure the nuanced semantic similarity between pairs of financial narratives. These narratives originate from the financial statements of the same company but correspond to different periods, such as year-over-year comparisons. Measuring the subtle semantic differences between these paired narratives enables market stakeholders to gauge changes over time in the company's financial and operational situations, which is critical for financial decision-making. We find that existing pretrained embedding models and LLM embeddings fall short in discerning these subtle financial narrative shifts. To address this gap, we propose an LLM-augmented pipeline specifically designed for the Financial-STS task. Evaluation on a human-annotated dataset demonstrates that our proposed method outperforms existing methods trained on classic STS tasks and generic LLM embeddings.