Beyond Cosine Similarity
This work provides a mathematically principled alternative to cosine similarity for semantic analysis, offering enhanced accuracy in complex embedding spaces, though it is incremental as it builds on existing similarity measurement frameworks.
The paper tackled the limitation of cosine similarity in capturing nonlinear semantic relationships by introducing recos, a new similarity metric based on a tighter upper bound for the dot product, which achieved higher correlation with human judgments on Semantic Textual Similarity benchmarks across 11 embedding models.
Cosine similarity, the standard metric for measuring semantic similarity in vector spaces, is mathematically grounded in the Cauchy-Schwarz inequality, which inherently limits it to capturing linear relationships--a constraint that fails to model the complex, nonlinear structures of real-world semantic spaces. We advance this theoretical underpinning by deriving a tighter upper bound for the dot product than the classical Cauchy-Schwarz bound. This new bound leads directly to recos, a similarity metric that normalizes the dot product by the sorted vector components. recos relaxes the condition for perfect similarity from strict linear dependence to ordinal concordance, thereby capturing a broader class of relationships. Extensive experiments across 11 embedding models--spanning static, contextualized, and universal types--demonstrate that recos consistently outperforms traditional cosine similarity, achieving higher correlation with human judgments on standard Semantic Textual Similarity (STS) benchmarks. Our work establishes recos as a mathematically principled and empirically superior alternative, offering enhanced accuracy for semantic analysis in complex embedding spaces.