CAffNet: Hard Constraint-Affine Neural Networks
For practitioners requiring guaranteed constraint satisfaction in neural networks (e.g., safety-critical applications), CAffNet provides a principled alternative to soft constraints or post-processing, though the novelty is incremental as it extends existing affine constraint methods.
CAffNet embeds hard affine constraints directly into neural network architectures via a trainable constraint-affine layer, enabling joint optimization with network parameters while guaranteeing constraint satisfaction for all inputs. The method preserves universal approximation properties and shows robust performance across diverse domains.
We present a novel framework for embedding hard constraint satisfaction into neural network (NN) architectures, specifically feedforward neural networks and transformers, with input-dependent affine constraints of arbitrary cardinality. Traditional constraint enforcement approaches either rely on penalty-based soft constraints, which offer no guarantee of satisfaction, or on post-processing methods that enforce constraints after the NN is trained, which may lead to suboptimality. We introduce a trainable constraint-affine (CAffine) layer into NNs, yielding CAffNet, which goes beyond enforcing affine constraints via fixed orthogonal or parallel projections and enables joint optimization with network parameters. Moreover, we impose no restrictions on the constraint space dimensions and establish that our construction preserves the universal approximation properties of NNs, while providing provable guarantees on constraint adherence for all inputs. Experimental validation demonstrates robust performance across diverse domains requiring guaranteed constraint satisfaction.