Crystal Twins: Self-supervised Learning for Crystalline Material Property Prediction
This work addresses the lack of large labeled datasets for material property prediction, offering a domain-specific solution that is incremental in applying self-supervised learning to crystalline materials.
The paper tackles the problem of predicting crystalline material properties by introducing Crystal Twins, a self-supervised learning method that pre-trains a Graph Neural Network on unlabeled data, resulting in significant performance improvements on 7 challenging benchmarks.
Machine learning (ML) models have been widely successful in the prediction of material properties. However, large labeled datasets required for training accurate ML models are elusive and computationally expensive to generate. Recent advances in Self-Supervised Learning (SSL) frameworks capable of training ML models on unlabeled data have mitigated this problem and demonstrated superior performance in computer vision and natural language processing tasks. Drawing inspiration from the developments in SSL, we introduce Crystal Twins (CT): an SSL method for crystalline materials property prediction. Using a large unlabeled dataset, we pre-train a Graph Neural Network (GNN) by applying the redundancy reduction principle to the graph latent embeddings of augmented instances obtained from the same crystalline system. By sharing the pre-trained weights when fine-tuning the GNN for regression tasks, we significantly improve the performance for 7 challenging material property prediction benchmarks