Cross-Fusion Distance: A Novel Metric for Measuring Fusion and Separability Between Data Groups in Representation Space
This addresses a fundamental problem in representation learning for researchers and practitioners dealing with domain shift, though it appears incremental as it builds on existing distributional distance metrics.
The paper tackles the problem of quantifying fusion and separability between data groups in representation space under domain shift, introducing Cross-Fusion Distance (CFD) which isolates fusion-altering geometry while remaining robust to fusion-preserving variations. CFD shows linear computational complexity and aligns more closely with downstream generalization degradation than existing metrics on real-world datasets.
Quantifying degrees of fusion and separability between data groups in representation space is a fundamental problem in representation learning, particularly under domain shift. A meaningful metric should capture fusion-altering factors like geometric displacement between representation groups, whose variations change the extent of fusion, while remaining invariant to fusion-preserving factors such as global scaling and sampling-induced layout changes, whose variations do not. Existing distributional distance metrics conflate these factors, leading to measures that are not informative of the true extent of fusion between data groups. We introduce Cross-Fusion Distance (CFD), a principled measure that isolates fusion-altering geometry while remaining robust to fusion-preserving variations, with linear computational complexity. We characterize the invariance and sensitivity properties of CFD theoretically and validate them in controlled synthetic experiments. For practical utility on real-world datasets with domain shift, CFD aligns more closely with downstream generalization degradation than commonly used alternatives. Overall, CFD provides a theoretically grounded and interpretable distance measure for representation learning.