Materials Discovery using Max K-Armed Bandit
This work addresses the problem of inefficient search algorithms for materials discovery, offering a more tailored approach, though it appears incremental as it builds on existing MKB methods.
The authors tackled the mismatch between conventional bandit algorithms and materials discovery goals by proposing a max K-armed bandit (MKB) algorithm that focuses on achieving the single best reward, which outperformed other bandit algorithms in late-stage searches when the optimal arm could not be determined by expected reward.
Search algorithms for the bandit problems are applicable in materials discovery. However, the objectives of the conventional bandit problem are different from those of materials discovery. The conventional bandit problem aims to maximize the total rewards, whereas materials discovery aims to achieve breakthroughs in material properties. The max K-armed bandit (MKB) problem, which aims to acquire the single best reward, matches with the discovery tasks better than the conventional bandit. Thus, here, we propose a search algorithm for materials discovery based on the MKB problem using a pseudo-value of the upper confidence bound of expected improvement of the best reward. This approach is pseudo-guaranteed to be asymptotic oracles that do not depends on the time horizon. In addition, compared with other MKB algorithms, the proposed algorithm has only one hyperparameter, which is advantageous in materials discovery. We applied the proposed algorithm to synthetic problems and molecular-design demonstrations using a Monte Carlo tree search. According to the results, the proposed algorithm stably outperformed other bandit algorithms in the late stage of the search process when the optimal arm of the MKB could not be determined based on its expectation reward.