Probing Classifiers: Promises, Shortcomings, and Advances
This is an incremental review that addresses methodological issues in interpreting NLP models, relevant for researchers in machine learning and natural language processing.
The article critically reviews the probing classifiers framework used for interpreting deep neural networks in NLP, highlighting its promises, methodological shortcomings, and recent advances without presenting new experimental results.
Probing classifiers have emerged as one of the prominent methodologies for interpreting and analyzing deep neural network models of natural language processing. The basic idea is simple -- a classifier is trained to predict some linguistic property from a model's representations -- and has been used to examine a wide variety of models and properties. However, recent studies have demonstrated various methodological limitations of this approach. This article critically reviews the probing classifiers framework, highlighting their promises, shortcomings, and advances.