Synergistic Benefits of Joint Molecule Generation and Property Prediction
This addresses the problem of integrating generative and predictive capabilities in molecular design for drug discovery applications, representing a novel method for a known bottleneck.
The authors tackled the challenge of training joint models for both molecule generation and property prediction by proposing Hyformer, a transformer-based model with alternating attention and joint pre-training. They demonstrated synergistic benefits in conditional sampling, out-of-distribution prediction, and representation learning, with application to discovering novel antimicrobial peptides.
Modeling the joint distribution of data samples and their properties allows to construct a single model for both data generation and property prediction, with synergistic benefits reaching beyond purely generative or predictive models. However, training joint models presents daunting architectural and optimization challenges. Here, we propose Hyformer, a transformer-based joint model that successfully blends the generative and predictive functionalities, using an alternating attention mechanism and a joint pre-training scheme. We show that Hyformer is simultaneously optimized for molecule generation and property prediction, while exhibiting synergistic benefits in conditional sampling, out-of-distribution property prediction and representation learning. Finally, we demonstrate the benefits of joint learning in a drug design use case of discovering novel antimicrobial~peptides.