Input Convex Lipschitz RNN: A Fast and Robust Approach for Engineering Tasks
This work addresses the need for fast and robust neural network models in engineering applications such as process optimization and control, representing an incremental improvement by integrating convexity and Lipschitz constraints.
The paper tackles the challenge of combining computational efficiency and robustness in neural networks for engineering tasks by introducing Input Convex Lipschitz Recurrent Neural Networks (ICLRNNs), which outperform existing recurrent units in both aspects and are applied to real-world scenarios like chemical process control and solar irradiance prediction.
Computational efficiency and robustness are essential in process modeling, optimization, and control for real-world engineering applications. While neural network-based approaches have gained significant attention in recent years, conventional neural networks often fail to address these two critical aspects simultaneously or even independently. Inspired by natural physical systems and established literature, input convex architectures are known to enhance computational efficiency in optimization tasks, whereas Lipschitz-constrained architectures improve robustness. However, combining these properties within a single model requires careful review, as inappropriate methods for enforcing one property can undermine the other. To overcome this, we introduce a novel network architecture, termed Input Convex Lipschitz Recurrent Neural Networks (ICLRNNs). This architecture seamlessly integrates the benefits of convexity and Lipschitz continuity, enabling fast and robust neural network-based modeling and optimization. The ICLRNN outperforms existing recurrent units in both computational efficiency and robustness. Additionally, it has been successfully applied to practical engineering scenarios, such as modeling and control of chemical process and the modeling and real-world solar irradiance prediction for solar PV system planning at LHT Holdings in Singapore. Source code is available at https://github.com/killingbear999/ICLRNN.