A Coding-Theoretic Analysis of Hyperspherical Prototypical Learning Geometry
This work provides a principled solution for hyperspherical prototypical learning, which is incremental but improves optimization and flexibility for representation learning tasks.
The paper tackles the problem of designing class prototypes on the unit hypersphere for representation learning by addressing unprincipled optimization and limited latent dimensions, resulting in near-optimal prototype placement with achievable and converse bounds.
Hyperspherical Prototypical Learning (HPL) is a supervised approach to representation learning that designs class prototypes on the unit hypersphere. The prototypes bias the representations to class separation in a scale invariant and known geometry. Previous approaches to HPL have either of the following shortcomings: (i) they follow an unprincipled optimisation procedure; or (ii) they are theoretically sound, but are constrained to only one possible latent dimension. In this paper, we address both shortcomings. To address (i), we present a principled optimisation procedure whose solution we show is optimal. To address (ii), we construct well-separated prototypes in a wide range of dimensions using linear block codes. Additionally, we give a full characterisation of the optimal prototype placement in terms of achievable and converse bounds, showing that our proposed methods are near-optimal.