Adjusting Interpretable Dimensions in Embedding Space with Human Judgments
This work addresses a specific bottleneck in natural language processing for researchers and practitioners by improving the accuracy of interpretable dimension extraction.
The paper tackled the problem of computing interpretable dimensions in embedding spaces by combining seed-based vectors with human ratings, resulting in markedly better performance, especially where seed-based methods fail.
Embedding spaces contain interpretable dimensions indicating gender, formality in style, or even object properties. This has been observed multiple times. Such interpretable dimensions are becoming valuable tools in different areas of study, from social science to neuroscience. The standard way to compute these dimensions uses contrasting seed words and computes difference vectors over them. This is simple but does not always work well. We combine seed-based vectors with guidance from human ratings of where words fall along a specific dimension, and evaluate on predicting both object properties like size and danger, and the stylistic properties of formality and complexity. We obtain interpretable dimensions with markedly better performance especially in cases where seed-based dimensions do not work well.