Bayesian Optimization in AlphaGo
This is an incremental application of Bayesian optimization to improve a specific AI system, AlphaGo, for Go enthusiasts and optimization practitioners.
The paper tackled hyperparameter tuning for AlphaGo using Bayesian optimization, resulting in a win-rate improvement from 50% to 66.5% in self-play games prior to the match with Lee Sedol.
During the development of AlphaGo, its many hyper-parameters were tuned with Bayesian optimization multiple times. This automatic tuning process resulted in substantial improvements in playing strength. For example, prior to the match with Lee Sedol, we tuned the latest AlphaGo agent and this improved its win-rate from 50% to 66.5% in self-play games. This tuned version was deployed in the final match. Of course, since we tuned AlphaGo many times during its development cycle, the compounded contribution was even higher than this percentage. It is our hope that this brief case study will be of interest to Go fans, and also provide Bayesian optimization practitioners with some insights and inspiration.