Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers
This addresses the problem of accurate molecular property prediction for computational chemistry, with incremental improvements in graph transformer architectures.
The paper tackles the limitation of graph transformers lacking third-order interactions for molecular geometry prediction by proposing the Triplet Graph Transformer (TGT), which achieves new state-of-the-art results on benchmarks like PCQM4Mv2 and OC20 IS2RE, with transfer learning yielding SOTA on QM9, MOLPCBA, and LIT-PCBA.
Graph transformers typically lack third-order interactions, limiting their geometric understanding which is crucial for tasks like molecular geometry prediction. We propose the Triplet Graph Transformer (TGT) that enables direct communication between pairs within a 3-tuple of nodes via novel triplet attention and aggregation mechanisms. TGT is applied to molecular property prediction by first predicting interatomic distances from 2D graphs and then using these distances for downstream tasks. A novel three-stage training procedure and stochastic inference further improve training efficiency and model performance. Our model achieves new state-of-the-art (SOTA) results on open challenge benchmarks PCQM4Mv2 and OC20 IS2RE. We also obtain SOTA results on QM9, MOLPCBA, and LIT-PCBA molecular property prediction benchmarks via transfer learning. We also demonstrate the generality of TGT with SOTA results on the traveling salesman problem (TSP).