GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction
This work addresses molecular property prediction for computational chemistry, but it is incremental as it builds on prior literature with optimizations.
The authors tackled molecular property prediction by developing GPS++, a hybrid MPNN/Transformer model that achieved a mean absolute error of 0.0719 on the PCQM4Mv2 test-challenge split, winning first place in the OGB-LSC 2022 challenge.
This technical report presents GPS++, the first-place solution to the Open Graph Benchmark Large-Scale Challenge (OGB-LSC 2022) for the PCQM4Mv2 molecular property prediction task. Our approach implements several key principles from the prior literature. At its core our GPS++ method is a hybrid MPNN/Transformer model that incorporates 3D atom positions and an auxiliary denoising task. The effectiveness of GPS++ is demonstrated by achieving 0.0719 mean absolute error on the independent test-challenge PCQM4Mv2 split. Thanks to Graphcore IPU acceleration, GPS++ scales to deep architectures (16 layers), training at 3 minutes per epoch, and large ensemble (112 models), completing the final predictions in 1 hour 32 minutes, well under the 4 hour inference budget allocated. Our implementation is publicly available at: https://github.com/graphcore/ogb-lsc-pcqm4mv2.