Geometry-Preserving Neural Architectures on Manifolds with Boundary
This work addresses the challenge of geometric structure preservation in machine learning for domains like dynamics and diffusion, representing an incremental advancement in neural architecture design.
The paper tackles the problem of preserving geometric structure in neural networks on manifolds with boundary by proposing a unified class of geometry-aware architectures that interleave geometric updates, establishing universal approximation results for constrained neural ODEs and analyzing output-enforced designs. The result includes exact feasibility for analytic updates and strong performance for learned projections in experiments on dynamics over S^2 and SO(3) and diffusion on S^{d-1}-valued features.
Preserving geometric structure is important in learning. We propose a unified class of geometry-aware architectures that interleave geometric updates between layers, where both projection layers and intrinsic exponential map updates arise as discretizations of projected dynamical systems on manifolds (with or without boundary). Within this framework, we establish universal approximation results for constrained neural ODEs. We also analyze architectures that enforce geometry only at the output, proving a separate universal approximation property that enables direct comparison to interleaved designs. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Experiments on dynamics over S^2 and SO(3), and diffusion on S^{d-1}-valued features demonstrate exact feasibility for analytic updates and strong performance for learned projections