Does My Representation Capture X? Probe-Ably
This work addresses the need for more reliable and accessible probing methodologies in machine learning research, though it is incremental as it builds on existing practices.
The authors tackled the challenge of ensuring reliable probing experiments in neural models by introducing Probe-Ably, a framework that automates and supports the application of best practices, simplifying the process for users.
Probing (or diagnostic classification) has become a popular strategy for investigating whether a given set of intermediate features is present in the representations of neural models. Probing studies may have misleading results, but various recent works have suggested more reliable methodologies that compensate for the possible pitfalls of probing. However, these best practices are numerous and fast-evolving. To simplify the process of running a set of probing experiments in line with suggested methodologies, we introduce Probe-Ably: an extendable probing framework which supports and automates the application of probing methods to the user's inputs.