Toward Robust Uncertainty Estimation with Random Activation Functions
This addresses uncertainty estimation in sensitive domains like healthcare and transportation, but it is incremental as it builds on existing ensemble methods.
The paper tackles the problem of inaccurate predictions in deep neural networks for out-of-distribution data by proposing a novel ensemble method using random activation functions to improve uncertainty quantification, and it demonstrates that this approach outperforms state-of-the-art methods on synthetic and real-world datasets in regression tasks.
Deep neural networks are in the limelight of machine learning with their excellent performance in many data-driven applications. However, they can lead to inaccurate predictions when queried in out-of-distribution data points, which can have detrimental effects especially in sensitive domains, such as healthcare and transportation, where erroneous predictions can be very costly and/or dangerous. Subsequently, quantifying the uncertainty of the output of a neural network is often leveraged to evaluate the confidence of its predictions, and ensemble models have proved to be effective in measuring the uncertainty by utilizing the variance of predictions over a pool of models. In this paper, we propose a novel approach for uncertainty quantification via ensembles, called Random Activation Functions (RAFs) Ensemble, that aims at improving the ensemble diversity toward a more robust estimation, by accommodating each neural network with a different (random) activation function. Extensive empirical study demonstrates that RAFs Ensemble outperforms state-of-the-art ensemble uncertainty quantification methods on both synthetic and real-world datasets in a series of regression tasks.