Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations
This provides a tool for analyzing complex multimodal neural network representations, though it appears incremental as an improvement over existing correlation detection methods.
The paper tackled the problem of detecting nonlinear correlations in high-dimensional multimodal representations by proposing a new metric based on intrinsic dimensionality entanglement, and demonstrated its effectiveness by uncovering clear correlations between visual and textual embeddings where existing methods failed.
To gain insight into the mechanisms behind machine learning methods, it is crucial to establish connections among the features describing data points. However, these correlations often exhibit a high-dimensional and strongly nonlinear nature, which makes them challenging to detect using standard methods. This paper exploits the entanglement between intrinsic dimensionality and correlation to propose a metric that quantifies the (potentially nonlinear) correlation between high-dimensional manifolds. We first validate our method on synthetic data in controlled environments, showcasing its advantages and drawbacks compared to existing techniques. Subsequently, we extend our analysis to large-scale applications in neural network representations. Specifically, we focus on latent representations of multimodal data, uncovering clear correlations between paired visual and textual embeddings, whereas existing methods struggle significantly in detecting similarity. Our results indicate the presence of highly nonlinear correlation patterns between latent manifolds.