Semantic Distance Measurement based on Multi-Kernel Gaussian Processes
This work addresses the need for adaptable semantic distance measures in computational linguistics, particularly for tasks like text retrieval and classification, though it appears incremental as it builds on existing Gaussian process and kernel methods.
The paper tackled the problem of semantic distance measurement by proposing a multi-kernel Gaussian process method that automatically learns kernel parameters from data, and demonstrated its effectiveness in fine-grained sentiment classification with large language models under in-context learning.
Semantic distance measurement is a fundamental problem in computational linguistics, providing a quantitative characterization of similarity or relatedness between text segments, and underpinning tasks such as text retrieval and text classification. From a mathematical perspective, a semantic distance can be viewed as a metric defined on a space of texts or on a representation space derived from them. However, most classical semantic distance methods are essentially fixed, making them difficult to adapt to specific data distributions and task requirements. In this paper, a semantic distance measure based on multi-kernel Gaussian processes (MK-GP) was proposed. The latent semantic function associated with texts was modeled as a Gaussian process, with its covariance function given by a combined kernel combining Matérn and polynomial components. The kernel parameters were learned automatically from data under supervision, rather than being hand-crafted. This semantic distance was instantiated and evaluated in the context of fine-grained sentiment classification with large language models under an in-context learning (ICL) setup. The experimental results demonstrated the effectiveness of the proposed measure.