Softmax as a Lagrangian-Legendrian Seam
This work bridges machine learning and differential geometry, offering a novel geometric interpretation of softmax, which could influence compact logit models and connections to information geometry.
The paper models the softmax function as a geometric interface between two potential-generated descriptions, showing that bias-shift invariance corresponds to Reeb flow and the Fenchel-Young equality provides a computable distance to this interface, with concrete examples for two- and three-class cases.
This note offers a first bridge from machine learning to modern differential geometry. We show that the logits-to-probabilities step implemented by softmax can be modeled as a geometric interface: two potential-generated, conservative descriptions (from negative entropy and log-sum-exp) meet along a Legendrian "seam" on a contact screen (the probability simplex) inside a simple folded symplectic collar. Bias-shift invariance appears as Reeb flow on the screen, and the Fenchel-Young equality/KL gap provides a computable distance to the seam. We work out the two- and three-class cases to make the picture concrete and outline next steps for ML: compact logit models (projective or spherical), global invariants, and connections to information geometry where on-screen dynamics manifest as replicator flows.