Combined Representation and Generation with Diffusive State Predictive Information Bottleneck
This work addresses the problem of expensive and rare data collection in molecular science by enabling compression and generation in a flexible architecture, though it appears incremental as it builds on existing methods like information bottlenecks and diffusion models.
The paper tackles the challenge of data-intensive generative modeling in high-dimensional molecular science by combining a time-lagged information bottleneck for representation learning with a diffusion model in a joint training objective, resulting in D-SPIB, which balances representation and generation and shows potential for exploring physical conditions beyond the training set.
Generative modeling becomes increasingly data-intensive in high-dimensional spaces. In molecular science, where data collection is expensive and important events are rare, compression to lower-dimensional manifolds is especially important for various downstream tasks, including generation. We combine a time-lagged information bottleneck designed to characterize molecular important representations and a diffusion model in one joint training objective. The resulting protocol, which we term Diffusive State Predictive Information Bottleneck (D-SPIB), enables the balancing of representation learning and generation aims in one flexible architecture. Additionally, the model is capable of combining temperature information from different molecular simulation trajectories to learn a coherent and useful internal representation of thermodynamics. We benchmark D-SPIB on multiple molecular tasks and showcase its potential for exploring physical conditions outside the training set.