Clarifying Trust of Materials Property Predictions using Neural Networks with Distribution-Specific Uncertainty Quantification
This work addresses the need for reliable uncertainty estimates in heterogeneous catalysis, an incremental step in applying UQ methods to a specific domain.
The study tackled the problem of ensuring trustworthy machine learning predictions for high-throughput catalyst discovery by evaluating uncertainty quantification (UQ) methods on a neural network predicting adsorption energies from the OC20 dataset, finding that evidential regression provides competitively trustworthy estimates and recalibration is essential for practical applications.
It is critical that machine learning (ML) model predictions be trustworthy for high-throughput catalyst discovery approaches. Uncertainty quantification (UQ) methods allow estimation of the trustworthiness of an ML model, but these methods have not been well explored in the field of heterogeneous catalysis. Herein, we investigate different UQ methods applied to a crystal graph convolutional neural network (CGCNN) to predict adsorption energies of molecules on alloys from the Open Catalyst 2020 (OC20) dataset, the largest existing heterogeneous catalyst dataset. We apply three UQ methods to the adsorption energy predictions, namely k-fold ensembling, Monte Carlo dropout, and evidential regression. The effectiveness of each UQ method is assessed based on accuracy, sharpness, dispersion, calibration, and tightness. Evidential regression is demonstrated to be a powerful approach for rapidly obtaining tunable, competitively trustworthy UQ estimates for heterogeneous catalysis applications when using neural networks. Recalibration of model uncertainties is shown to be essential in practical screening applications of catalysts using uncertainties.