Multi-fidelity regression using artificial neural networks: efficient approximation of parameter-dependent output quantities
This work addresses data scarcity issues in engineering and scientific simulations by improving multi-fidelity regression methods, offering a more efficient way to approximate parameter-dependent outputs, though it is incremental as it builds on existing approaches.
The paper tackles the problem of limited high-fidelity data in numerical or physical experiments by proposing new neural network architectures for multi-fidelity regression, which leverage correlations between high- and low-fidelity data sources. The results show that these neural network models consistently outperform traditional co-kriging schemes on benchmarks and achieve comparable accuracy to expensive full-order models using only very few high-fidelity evaluations combined with many cheap low-fidelity ones in an engineering application.
Highly accurate numerical or physical experiments are often time-consuming or expensive to obtain. When time or budget restrictions prohibit the generation of additional data, the amount of available samples may be too limited to provide satisfactory model results. Multi-fidelity methods deal with such problems by incorporating information from other sources, which are ideally well-correlated with the high-fidelity data, but can be obtained at a lower cost. By leveraging correlations between different data sets, multi-fidelity methods often yield superior generalization when compared to models based solely on a small amount of high-fidelity data. In this work, we present the use of artificial neural networks applied to multi-fidelity regression problems. By elaborating a few existing approaches, we propose new neural network architectures for multi-fidelity regression. The introduced models are compared against a traditional multi-fidelity scheme, co-kriging. A collection of artificial benchmarks are presented to measure the performance of the analyzed models. The results show that cross-validation in combination with Bayesian optimization consistently leads to neural network models that outperform the co-kriging scheme. Additionally, we show an application of multi-fidelity regression to an engineering problem. The propagation of a pressure wave into an acoustic horn with parametrized shape and frequency is considered, and the index of reflection intensity is approximated using the multi-fidelity models. A finite element model and a reduced basis model are adopted as the high- and low-fidelity, respectively. It is shown that the multi-fidelity neural network returns outputs that achieve a comparable accuracy to those from the expensive, full-order model, using only very few full-order evaluations combined with a larger amount of inaccurate but cheap evaluations of a reduced order model.