Robust Bayesian Optimization with Student-t Likelihood
This work addresses a specific issue in hyperparameter tuning for machine learning, offering an incremental improvement to enhance robustness against outliers in Bayesian optimization.
The authors tackled the problem of Bayesian optimization being sensitive to outliers by developing a Gaussian process model with a Student-t likelihood to robustly handle outliers and improve optimization efficiency. They demonstrated the method's effectiveness through numerical results on artificial functions and real-world problems.
Bayesian optimization has recently attracted the attention of the automatic machine learning community for its excellent results in hyperparameter tuning. BO is characterized by the sample efficiency with which it can optimize expensive black-box functions. The efficiency is achieved in a similar fashion to the learning to learn methods: surrogate models (typically in the form of Gaussian processes) learn the target function and perform intelligent sampling. This surrogate model can be applied even in the presence of noise; however, as with most regression methods, it is very sensitive to outlier data. This can result in erroneous predictions and, in the case of BO, biased and inefficient exploration. In this work, we present a GP model that is robust to outliers which uses a Student-t likelihood to segregate outliers and robustly conduct Bayesian optimization. We present numerical results evaluating the proposed method in both artificial functions and real problems.