Bayesian Optimization for Parameter Tuning of the XOR Neural Network
This work addresses parameter tuning challenges for neural network users, but it is incremental as it applies an existing method to a specific case.
The paper tackled the problem of automating parameter tuning for neural networks, particularly when limited search attempts are available, by implementing Bayesian Optimization with Gaussian Process Priors to tune an XOR neural network, resulting in higher prediction accuracy.
When applying Machine Learning techniques to problems, one must select model parameters to ensure that the system converges but also does not become stuck at the objective function's local minimum. Tuning these parameters becomes a non-trivial task for large models and it is not always apparent if the user has found the optimal parameters. We aim to automate the process of tuning a Neural Network, (where only a limited number of parameter search attempts are available) by implementing Bayesian Optimization. In particular, by assigning Gaussian Process Priors to the parameter space, we utilize Bayesian Optimization to tune an Artificial Neural Network used to learn the XOR function, with the result of achieving higher prediction accuracy.