Assessing the Extrapolation Capability of Template-Free Retrosynthesis Models
This work addresses the problem of evaluating template-free models for retrosynthesis prediction in chemistry, revealing significant limitations in their ability to generate plausible novel reactions, which is incremental as it builds on existing research.
The study assessed the extrapolation capability of state-of-the-art template-free models for retrosynthesis prediction, finding that their top-10 exact-match accuracy in out-of-distribution reactions is less than 1% and over half of the novel reactions predicted are chemically implausible.
Despite the acknowledged capability of template-free models in exploring unseen reaction spaces compared to template-based models for retrosynthesis prediction, their ability to venture beyond established boundaries remains relatively uncharted. In this study, we empirically assess the extrapolation capability of state-of-the-art template-free models by meticulously assembling an extensive set of out-of-distribution (OOD) reactions. Our findings demonstrate that while template-free models exhibit potential in predicting precursors with novel synthesis rules, their top-10 exact-match accuracy in OOD reactions is strikingly modest (< 1%). Furthermore, despite the capability of generating novel reactions, our investigation highlights a recurring issue where more than half of the novel reactions predicted by template-free models are chemically implausible. Consequently, we advocate for the future development of template-free models that integrate considerations of chemical feasibility when navigating unexplored regions of reaction space.