Binary Diffusion Probabilistic Model
This work addresses the challenge of applying diffusion models to discrete binary representations, which is incremental as it adapts existing methods to a specific data type.
The authors tackled the problem of adapting diffusion models to binary data by proposing the Binary Diffusion Probabilistic Model (BDPM), which outperformed state-of-the-art methods on image-to-image translation tasks like super-resolution and achieved competitive results on ImageNet-1k with low parameter counts and few sampling steps.
We propose the Binary Diffusion Probabilistic Model (BDPM), a generative framework specifically designed for data representations in binary form. Conventional denoising diffusion probabilistic models (DDPMs) assume continuous inputs, use mean squared error objectives and Gaussian perturbations, i.e., assumptions that are not suited to discrete and binary representations. BDPM instead encodes images into binary representations using multi bit-plane and learnable binary embeddings, perturbs them via XOR-based noise, and trains a model by optimizing a binary cross-entropy loss. These binary representations offer fine-grained noise control, accelerate convergence, and reduce inference cost. On image-to-image translation tasks, such as super-resolution, inpainting, and blind restoration, BDPM based on a small denoiser and multi bit-plane representation outperforms state-of-the-art methods on FFHQ, CelebA, and CelebA-HQ using a few sampling steps. In class-conditional generation on ImageNet-1k, BDPM based on learnable binary embeddings achieves competitive results among models with both low parameter counts and a few sampling steps.