Geometric Second-Order Feature Correlation Learning for Self-Supervised Speech Emotion Recognition
For speech emotion recognition researchers, this work addresses the bottleneck of aggregating SSL features by preserving geometric relationships, offering a novel method for improved performance.
Self-supervised speech emotion recognition suffers from first-order aggregation that ignores feature correlations. The proposed Second-Order Correlation (SOC) layer models covariance descriptors via Log-Euclidean mapping, achieving state-of-the-art results on ESD and RAVDESS datasets.
Self-supervised learning (SSL) yields powerful, context-rich representations for speech emotion recognition (SER), yet aggregating these representations into holistic descriptors remains a bottleneck. Conventional first-order aggregation implicitly assumes feature independence, which overlooks the latent Riemannian geometry and discards higher-order relationships essential to the representational power of the backbone. To address this problem, this paper proposes a novel Second-Order Correlation (SOC) layer. Instead of treating features in isolation, SOC models feature correlations as covariance descriptors to capture synergistic co-occurrence patterns, which serve as discriminative signatures for robust emotion recognition. By mapping these descriptors from the Riemannian manifold to a Euclidean tangent space through Log-Euclidean mapping (LEM), the proposed method preserves geometric integrity while enabling direct linear discriminative learning. Extensive experiments on the ESD and RAVDESS datasets demonstrate that SOC recovers discriminative information lost in first-order pooling and effectively aggregates high-dimensional SSL features.