Jacobian-Enhanced Neural Networks
This work addresses the need for efficient gradient-enhanced surrogate models in computer-aided design, offering incremental improvements for optimization tasks.
The paper tackles the problem of creating fast and accurate surrogate models for computer-aided design by introducing Jacobian-Enhanced Neural Networks (JENN), which improve accuracy with fewer training points and provide accurate partial derivatives, demonstrating superiority over standard neural networks in surrogate-based optimization.
Jacobian-Enhanced Neural Networks (JENN) are densely connected multi-layer perceptrons, whose training process is modified to predict partial derivatives accurately. Their main benefit is better accuracy with fewer training points compared to standard neural networks. These attributes are particularly desirable in the field of computer-aided design, where there is often the need to replace computationally expensive, physics-based models with fast running approximations, known as surrogate models or meta-models. Since a surrogate emulates the original model accurately in near-real time, it yields a speed benefit that can be used to carry out orders of magnitude more function calls quickly. However, in the special case of gradient-enhanced methods, there is the additional value proposition that partial derivatives are accurate, which is a critical property for one important use-case: surrogate-based optimization. This work derives the complete theory and exemplifies its superiority over standard neural nets for surrogate-based optimization.