Towards Physics-Guided Foundation Models
This addresses the issue of unreliable model outputs in scientific and general domains, but it appears incremental as it builds on existing foundation model concepts by adding physics guidance.
The paper tackles the problem of traditional foundation models producing unrealistic and physically infeasible outputs, especially for out-of-distribution prediction, by proposing physics-guided foundation models (PGFM) that integrate broad physical knowledge to improve applicability across downstream tasks.
Traditional foundation models are pre-trained on broad datasets to reduce the training resources (e.g., time, energy, labeled samples) needed for fine-tuning a wide range of downstream tasks. However, traditional foundation models struggle with out-of-distribution prediction and can produce outputs that are unrealistic and physically infeasible. We propose the notation of physics-guided foundation models (PGFM), that is, foundation models integrated with broad or general domain (e.g., scientific) physical knowledge applicable to a wide range of downstream tasks.