Idioms, Probing and Dangerous Things: Towards Structural Probing for Idiomaticity in Vector Space
This addresses the problem of understanding idiomaticity in NLP for researchers, but it is incremental as it builds on existing probing methods and datasets.
The paper investigates how idiomatic information is structurally encoded in embeddings by applying a structural probing method to static (GloVe) and contextual (BERT) embeddings using a repurposed English verbal multi-word expression dataset, finding that both encode some idiomatic information but with conflicting evidence on whether it relates to vector norms.
The goal of this paper is to learn more about how idiomatic information is structurally encoded in embeddings, using a structural probing method. We repurpose an existing English verbal multi-word expression (MWE) dataset to suit the probing framework and perform a comparative probing study of static (GloVe) and contextual (BERT) embeddings. Our experiments indicate that both encode some idiomatic information to varying degrees, but yield conflicting evidence as to whether idiomaticity is encoded in the vector norm, leaving this an open question. We also identify some limitations of the used dataset and highlight important directions for future work in improving its suitability for a probing analysis.