Data-Dependent Smoothing for Protein Discovery with Walk-Jump Sampling
This work addresses protein discovery for computational biology, offering an incremental improvement by incorporating local data geometry into generative modeling.
The paper tackled the problem of generating protein data with diffusion models by addressing the uneven sparsity in data distributions, introducing a data-dependent smoothing method that improved performance across multiple metrics.
Diffusion models have emerged as a powerful class of generative models by learning to iteratively reverse the noising process. Their ability to generate high-quality samples has extended beyond high-dimensional image data to other complex domains such as proteins, where data distributions are typically sparse and unevenly spread. Importantly, the sparsity itself is uneven. Empirically, we observed that while a small fraction of samples lie in dense clusters, the majority occupy regions of varying sparsity across the data space. Existing approaches largely ignore this data-dependent variability. In this work, we introduce a Data-Dependent Smoothing Walk-Jump framework that employs kernel density estimation (KDE) as a preprocessing step to estimate the noise scale $σ$ for each data point, followed by training a score model with these data-dependent $σ$ values. By incorporating local data geometry into the denoising process, our method accounts for the heterogeneous distribution of protein data. Empirical evaluations demonstrate that our approach yields consistent improvements across multiple metrics, highlighting the importance of data-aware sigma prediction for generative modeling in sparse, high-dimensional settings.