OOD-Probe: A Neural Interpretation of Out-of-Domain Generalization
This provides insights for researchers developing OOD generalization systems, though it is incremental as it builds on existing evaluation methods.
The paper tackles the problem of understanding out-of-domain (OOD) generalization in deep neural networks by proposing a probing framework to analyze domain information in intermediate representations, finding that encoding patterns vary by dataset and correlate with generalization performance.
The ability to generalize out-of-domain (OOD) is an important goal for deep neural network development, and researchers have proposed many high-performing OOD generalization methods from various foundations. While many OOD algorithms perform well in various scenarios, these systems are evaluated as ``black-boxes''. Instead, we propose a flexible framework that evaluates OOD systems with finer granularity using a probing module that predicts the originating domain from intermediate representations. We find that representations always encode some information about the domain. While the layerwise encoding patterns remain largely stable across different OOD algorithms, they vary across the datasets. For example, the information about rotation (on RotatedMNIST) is the most visible on the lower layers, while the information about style (on VLCS and PACS) is the most visible on the middle layers. In addition, the high probing results correlate to the domain generalization performances, leading to further directions in developing OOD generalization systems.