Conditional probing: measuring usable information beyond a baseline
This addresses the issue of accurately assessing usable information in representations for NLP researchers, though it is incremental as it builds on existing probing theory.
The paper tackles the problem of measuring information in neural representations beyond baseline comparisons by proposing conditional probing, which conditions on baseline information; it finds that properties like part-of-speech are accessible at deeper network layers than previously thought after conditioning on non-contextual word embeddings.
Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable. One suggests that a representation encodes a property if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embeddings. Instead of using baselines as a point of comparison, we're interested in measuring information that is contained in the representation but not in the baseline. For example, current methods can detect when a representation is more useful than the word identity (a baseline) for predicting part-of-speech; however, they cannot detect when the representation is predictive of just the aspects of part-of-speech not explainable by the word identity. In this work, we extend a theory of usable information called $\mathcal{V}$-information and propose conditional probing, which explicitly conditions on the information in the baseline. In a case study, we find that after conditioning on non-contextual word embeddings, properties like part-of-speech are accessible at deeper layers of a network than previously thought.