Residual Semantic Decomposition of Word Embeddings
Provides a qualitative diagnostic tool for analyzing semantic structure in word embeddings, but results are incremental and not benchmark-level.
The authors propose Residual Semantic Decomposition (RSD) to decompose word embeddings into semantic axes, showing it can separate context anchors above chance but finds that ambiguous words are not uniformly high-entropy boundary points in static GloVe.
We introduce Residual Semantic Decomposition (RSD), a neural additive decomposition of word embeddings that balances embedding reconstruction with relational structure preservation. RSD supports recursive binary decomposition: each $K=2$ fit extracts a local semantic axis, while residuals expose information not absorbed by that axis. In manually specified paired-context diagnostics over ambiguous words, RSD separates supplied context anchors above shuffled-label controls, but entropy diagnostics show that ambiguous targets are not uniformly high-entropy boundary points in static GloVe. We therefore treat residual neighborhoods as qualitative diagnostics rather than benchmark sense predictions.