Unsupervised physics-informed disentanglement of multimodal data for high-throughput scientific discovery
This work addresses bottlenecks in high-throughput scientific discovery for materials manufacturing, but it is incremental as it builds on existing variational inference and multimodal fusion techniques.
The authors tackled the problem of discovering shared information in multimodal scientific datasets by introducing physics-informed multimodal autoencoders (PIMA), which enabled accurate cross-modal inference between images of mesoscale topology and mechanical stress-strain response in a dataset of lattice metamaterials from metal additive manufacturing.
We introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal scientific datasets representative of high-throughput testing. Individual modalities are embedded into a shared latent space and fused through a product of experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture of expert decoder imposing inductive biases encoding prior scientific knowledge and imparting structured disentanglement of the latent space. This approach enables discovery of fingerprints which may be detected in high-dimensional heterogeneous datasets, avoiding traditional bottlenecks related to high-fidelity measurement and characterization. Motivated by accelerated co-design and optimization of materials manufacturing processes, a dataset of lattice metamaterials from metal additive manufacturing demonstrates accurate cross modal inference between images of mesoscale topology and mechanical stress-strain response.