Designing Biological Sequences via Meta-Reinforcement Learning and Bayesian Optimization
This work addresses the challenge of accelerating biological sequence design for medical applications, presenting an incremental improvement over existing methods.
The paper tackles the problem of designing biological sequences by framing it as a global optimization with an expensive black-box function, proposing MetaRLBO which uses meta-reinforcement learning to train a generative model for sequence selection via Bayesian Optimization, achieving competitive results in in-silico experiments.
The ability to accelerate the design of biological sequences can have a substantial impact on the progress of the medical field. The problem can be framed as a global optimization problem where the objective is an expensive black-box function such that we can query large batches restricted with a limitation of a low number of rounds. Bayesian Optimization is a principled method for tackling this problem. However, the astronomically large state space of biological sequences renders brute-force iterating over all possible sequences infeasible. In this paper, we propose MetaRLBO where we train an autoregressive generative model via Meta-Reinforcement Learning to propose promising sequences for selection via Bayesian Optimization. We pose this problem as that of finding an optimal policy over a distribution of MDPs induced by sampling subsets of the data acquired in the previous rounds. Our in-silico experiments show that meta-learning over such ensembles provides robustness against reward misspecification and achieves competitive results compared to existing strong baselines.