Conservative objective models are a special kind of contrastive divergence-based energy model
This work addresses theoretical understanding and sampling improvements for offline optimization models, but it appears incremental as it builds on existing COM frameworks.
The paper shows that conservative objective models (COMs) for offline model-based optimization are a special case of contrastive divergence-based energy models, and proposes a fix using Langevin MCMC sampling to improve sample quality by decoupling unconditional and conditional probabilities.
In this work we theoretically show that conservative objective models (COMs) for offline model-based optimisation (MBO) are a special kind of contrastive divergence-based energy model, one where the energy function represents both the unconditional probability of the input and the conditional probability of the reward variable. While the initial formulation only samples modes from its learned distribution, we propose a simple fix that replaces its gradient ascent sampler with a Langevin MCMC sampler. This gives rise to a special probabilistic model where the probability of sampling an input is proportional to its predicted reward. Lastly, we show that better samples can be obtained if the model is decoupled so that the unconditional and conditional probabilities are modelled separately.