Cardinality-Preserving Structured Sparse Graph Transformers for Molecular Property Prediction
This work addresses data-efficient drug discovery for pharmaceutical research, representing an incremental improvement over existing graph transformer methods.
The paper tackled molecular property prediction with limited labeled data by introducing CardinalGraphFormer, a graph transformer with structured sparse attention and cardinality-preserving aggregation, which improved mean performance across 11 tasks and achieved statistically significant gains on 10 benchmarks.
Drug discovery motivates efficient molecular property prediction under limited labeled data. Chemical space is vast, often estimated at approximately 10^60 drug-like molecules, while only thousands of drugs have been approved. As a result, self-supervised pretraining on large unlabeled molecular corpora has become essential for data-efficient molecular representation learning. We introduce **CardinalGraphFormer**, a graph transformer that incorporates Graphormer-inspired structural biases, including shortest-path distance and centrality, as well as direct-bond edge bias, within a structured sparse attention regime limited to shortest-path distance <= 3. The model further augments this design with a cardinality-preserving unnormalized aggregation channel over the same support set. Pretraining combines contrastive graph-level alignment with masked attribute reconstruction. Under a fully matched evaluation protocol, CardinalGraphFormer improves mean performance across all 11 evaluated tasks and achieves statistically significant gains on 10 of 11 public benchmarks spanning MoleculeNet, OGB, and TDC ADMET tasks when compared to strong reproduced baselines.