Robust Hyperbolic Learning with Curvature-Aware Optimization
This work addresses robustness and efficiency problems in hyperbolic learning for computer vision and related domains, representing an incremental improvement with specific gains.
The paper tackles the issues of overfitting, computational expense, and instability in hyperbolic deep learning by introducing a Riemannian AdamW derivation and a fine-tunable hyperbolic scaling approach, resulting in state-of-the-art performance in two domains and drastically reduced runtime.
Hyperbolic deep learning has become a growing research direction in computer vision due to the unique properties afforded by the alternate embedding space. The negative curvature and exponentially growing distance metric provide a natural framework for capturing hierarchical relationships between datapoints and allowing for finer separability between their embeddings. However, current hyperbolic learning approaches are still prone to overfitting, computationally expensive, and prone to instability, especially when attempting to learn the manifold curvature to adapt to tasks and different datasets. To address these issues, our paper presents a derivation for Riemannian AdamW that helps increase hyperbolic generalization ability. For improved stability, we introduce a novel fine-tunable hyperbolic scaling approach to constrain hyperbolic embeddings and reduce approximation errors. Using this along with our curvature-aware learning schema for Lorentzian Optimizers enables the combination of curvature and non-trivialized hyperbolic parameter learning. Our approach demonstrates consistent performance improvements across Computer Vision, EEG classification, and hierarchical metric learning tasks achieving state-of-the-art results in two domains and drastically reducing runtime.