MetaDNS: Enhancing Exploration in Discrete Neural Samplers via Well-Tempered Metadynamics
For researchers in machine learning and computational physics, MetaDNS provides a general framework to enhance exploration in discrete neural samplers, addressing a key limitation of existing methods.
MetaDNS integrates well-tempered metadynamics into discrete neural samplers to overcome mode collapse and explore high-energy barrier regions, enabling accurate free energy estimation on challenging low-temperature benchmarks like Ising, Potts, and copper-gold alloy.
Sampling from discrete distributions with multiple modes and energy barriers is fundamental to machine learning and computational physics. Recent discrete neural samplers like MDNS suffer from mode collapse and fail to sample high-energy barrier regions between modes, which is critical for free energy estimation and understanding phase transitions. We propose Metadynamics Discrete Neural Sampler (MetaDNS), a general framework integrating well-tempered metadynamics into discrete diffusion or autoregressive samplers. By maintaining an adaptive, history-dependent bias potential along selected low-dimensional coordinates, MetaDNS forces exploration of previously inaccessible regions, enabling free energy reconstruction infeasible with standard neural samplers due to a lack of high-energy samples. On challenging low-temperature benchmarks including Ising, Potts, and the copper-gold binary alloy, MetaDNS reproduces the thermodynamic distribution. Compared to MCMC-based metadynamics, MetaDNS also achieves comparable exploration requiring fewer bias deposition steps.