Bridging Continuous and Discrete Spaces: Interpretable Sentence Representation Learning via Compositional Operations
This work addresses the need for interpretable sentence representations in NLP, offering a novel framework that bridges continuous and discrete spaces, though it is incremental in building on existing embedding methods.
The paper tackled the problem of making sentence embeddings interpretable via compositional operations like fusion or difference, and showed that their method InterSent significantly improves interpretability on four textual generation tasks while maintaining strong performance on semantic similarity tasks.
Traditional sentence embedding models encode sentences into vector representations to capture useful properties such as the semantic similarity between sentences. However, in addition to similarity, sentence semantics can also be interpreted via compositional operations such as sentence fusion or difference. It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space. To more effectively bridge the continuous embedding and discrete text spaces, we explore the plausibility of incorporating various compositional properties into the sentence embedding space that allows us to interpret embedding transformations as compositional sentence operations. We propose InterSent, an end-to-end framework for learning interpretable sentence embeddings that supports compositional sentence operations in the embedding space. Our method optimizes operator networks and a bottleneck encoder-decoder model to produce meaningful and interpretable sentence embeddings. Experimental results demonstrate that our method significantly improves the interpretability of sentence embeddings on four textual generation tasks over existing approaches while maintaining strong performance on traditional semantic similarity tasks.