Soft-Radial Projection for Constrained End-to-End Learning
This addresses a fundamental bottleneck for integrating hard constraints into deep learning, which is essential for safety-critical systems.
The paper tackles the problem of gradient saturation in deep learning layers that enforce hard constraints through projection onto constraint boundaries, which causes optimization stalls. They introduce Soft-Radial Projection, a differentiable layer that maps points into the feasible set interior while preserving full-rank Jacobians, resulting in improved convergence and solution quality over existing methods.
Integrating hard constraints into deep learning is essential for safety-critical systems. Yet existing constructive layers that project predictions onto constraint boundaries face a fundamental bottleneck: gradient saturation. By collapsing exterior points onto lower-dimensional surfaces, standard orthogonal projections induce rank-deficient Jacobians, which nullify gradients orthogonal to active constraints and hinder optimization. We introduce Soft-Radial Projection, a differentiable reparameterization layer that circumvents this issue through a radial mapping from Euclidean space into the interior of the feasible set. This construction guarantees strict feasibility while preserving a full-rank Jacobian almost everywhere, thereby preventing the optimization stalls typical of boundary-based methods. We theoretically prove that the architecture retains the universal approximation property and empirically show improved convergence behavior and solution quality over state-of-the-art optimization- and projection-based baselines.