A Lie Group Approach to Riemannian Batch Normalization
This work addresses the problem of inconsistent normalization methods for manifold-valued data in machine learning, offering a more generalizable solution for domains like computer vision and signal processing, though it is incremental as it builds on existing Riemannian normalization techniques.
The paper tackles the lack of a unified framework for Riemannian Batch Normalization on manifolds by proposing a Lie group-based approach, achieving improved performance in tasks like radar recognition, human action recognition, and EEG classification with concrete experimental results.
Manifold-valued measurements exist in numerous applications within computer vision and machine learning. Recent studies have extended Deep Neural Networks (DNNs) to manifolds, and concomitantly, normalization techniques have also been adapted to several manifolds, referred to as Riemannian normalization. Nonetheless, most of the existing Riemannian normalization methods have been derived in an ad hoc manner and only apply to specific manifolds. This paper establishes a unified framework for Riemannian Batch Normalization (RBN) techniques on Lie groups. Our framework offers the theoretical guarantee of controlling both the Riemannian mean and variance. Empirically, we focus on Symmetric Positive Definite (SPD) manifolds, which possess three distinct types of Lie group structures. Using the deformation concept, we generalize the existing Lie groups on SPD manifolds into three families of parameterized Lie groups. Specific normalization layers induced by these Lie groups are then proposed for SPD neural networks. We demonstrate the effectiveness of our approach through three sets of experiments: radar recognition, human action recognition, and electroencephalography (EEG) classification. The code is available at https://github.com/GitZH-Chen/LieBN.git.