Neural Fields with Hard Constraints of Arbitrary Differential Order
This addresses a bottleneck in deep learning for optimization problems where explicit constraint satisfaction is needed, offering a domain-specific solution.
The paper tackles the problem of enforcing hard constraints on neural networks during optimization, developing Constrained Neural Fields (CNF) to handle constraints as linear operators on the neural field and its derivatives, with demonstrations in real-world applications.
While deep learning techniques have become extremely popular for solving a broad range of optimization problems, methods to enforce hard constraints during optimization, particularly on deep neural networks, remain underdeveloped. Inspired by the rich literature on meshless interpolation and its extension to spectral collocation methods in scientific computing, we develop a series of approaches for enforcing hard constraints on neural fields, which we refer to as Constrained Neural Fields (CNF). The constraints can be specified as a linear operator applied to the neural field and its derivatives. We also design specific model representations and training strategies for problems where standard models may encounter difficulties, such as conditioning of the system, memory consumption, and capacity of the network when being constrained. Our approaches are demonstrated in a wide range of real-world applications. Additionally, we develop a framework that enables highly efficient model and constraint specification, which can be readily applied to any downstream task where hard constraints need to be explicitly satisfied during optimization.