Large language models, physics-based modeling, experimental measurements: the trinity of data-scarce learning of polymer properties
This addresses the challenge of costly and limited experimental data for material scientists, though it is incremental as it builds on existing methods with a hybrid approach.
The paper tackles the problem of data scarcity in training large language models for material modeling by introducing a physics-based training pipeline that uses synthetic data for pretraining before finetuning with limited experimental data, resulting in improved accuracy for learning polymer flammability metrics.
Large language models (LLMs) bear promise as a fast and accurate material modeling paradigm for evaluation, analysis, and design. Their vast number of trainable parameters necessitates a wealth of data to achieve accuracy and mitigate overfitting. However, experimental measurements are often limited and costly to obtain in sufficient quantities for finetuning. To this end, we present a physics-based training pipeline that tackles the pathology of data scarcity. The core enabler is a physics-based modeling framework that generates a multitude of synthetic data to align the LLM to a physically consistent initial state before finetuning. Our framework features a two-phase training strategy: (1) utilizing the large-in-amount while less accurate synthetic data for supervised pretraining, and (2) finetuning the phase-1 model with limited experimental data. We empirically demonstrate that supervised pretraining is vital to obtaining accurate finetuned LLMs, via the lens of learning polymer flammability metrics where cone calorimeter data is sparse.