Feature Learning in Image Hierarchies using Functional Maximal Correlation
This addresses the challenge of interpretability and efficiency in self-supervised learning for image hierarchies, though it appears incremental as it builds on existing CNN architectures.
The paper tackles the problem of characterizing dependencies across hierarchical levels in multiview image systems by proposing HFMCA, which achieves faster convergence and increased stability in self-supervised learning through orthonormal basis functions.
This paper proposes the Hierarchical Functional Maximal Correlation Algorithm (HFMCA), a hierarchical methodology that characterizes dependencies across two hierarchical levels in multiview systems. By framing view similarities as dependencies and ensuring contrastivity by imposing orthonormality, HFMCA achieves faster convergence and increased stability in self-supervised learning. HFMCA defines and measures dependencies within image hierarchies, from pixels and patches to full images. We find that the network topology for approximating orthonormal basis functions aligns with a vanilla CNN, enabling the decomposition of density ratios between neighboring layers of feature maps. This approach provides powerful interpretability, revealing the resemblance between supervision and self-supervision through the lens of internal representations.