Wasserstein Distributionally Robust Shallow Convex Neural Networks
This work addresses safety concerns in neural networks for critical domains like energy, though it is incremental as it builds on existing convex and distributionally robust optimization methods.
The authors tackled the problem of making neural networks more reliable for critical applications by proposing Wasserstein distributionally robust shallow convex neural networks, which provide out-of-sample performance guarantees and enforce hard convex physical constraints, demonstrated through synthetic and real-world power system experiments.
In this work, we propose Wasserstein distributionally robust shallow convex neural networks (WaDiRo-SCNNs) to provide reliable nonlinear predictions when subject to adverse and corrupted datasets. Our approach is based on the reformulation of a new convex training program for ReLU-based shallow neural networks, which allows us to cast the problem into the order-1 Wasserstein distributionally robust optimization framework. Our training procedure is conservative, has low stochasticity, is solvable with open-source solvers, and is scalable to large industrial deployments. We provide out-of-sample performance guarantees, show that hard convex physical constraints can be enforced in the training program, and propose a mixed-integer convex post-training verification program to evaluate model stability. WaDiRo-SCNN aims to make neural networks safer for critical applications, such as in the energy sector. Finally, we numerically demonstrate our model's performance through both a synthetic experiment and a real-world power system application, viz., the prediction of hourly energy consumption in non-residential buildings within the context of virtual power plants, and evaluate its stability across standard regression benchmark datasets. The experimental results are convincing and showcase the strengths of the proposed model.