RetroOOD: Understanding Out-of-Distribution Generalization in Retrosynthesis Prediction
This addresses the challenge of real-world deployment for retrosynthesis prediction models, which is incremental as it builds on existing methods to improve OOD generalization.
The paper tackles the problem of performance degradation in retrosynthesis prediction models when faced with out-of-distribution molecules or reactions, by constructing benchmark datasets and proposing two model-agnostic techniques that achieve an average performance improvement of 4.6%.
Machine learning-assisted retrosynthesis prediction models have been gaining widespread adoption, though their performances oftentimes degrade significantly when deployed in real-world applications embracing out-of-distribution (OOD) molecules or reactions. Despite steady progress on standard benchmarks, our understanding of existing retrosynthesis prediction models under the premise of distribution shifts remains stagnant. To this end, we first formally sort out two types of distribution shifts in retrosynthesis prediction and construct two groups of benchmark datasets. Next, through comprehensive experiments, we systematically compare state-of-the-art retrosynthesis prediction models on the two groups of benchmarks, revealing the limitations of previous in-distribution evaluation and re-examining the advantages of each model. More remarkably, we are motivated by the above empirical insights to propose two model-agnostic techniques that can improve the OOD generalization of arbitrary off-the-shelf retrosynthesis prediction algorithms. Our preliminary experiments show their high potential with an average performance improvement of 4.6%, and the established benchmarks serve as a foothold for further retrosynthesis prediction research towards OOD generalization.