Sascha Holl

Jente Vandersanden, Sascha Holl, Xingchang Huang et al.

Classical diffusion models typically rely on isotropic Gaussian noise, treating all regions uniformly and overlooking structural information important for high-quality generation. We introduce an edge-preserving diffusion process that generalizes isotropic models via a hybrid noise scheme with an edge-aware scheduler that smoothly transitions from edge-preserving to isotropic noise. This enables the model to capture fine structural details while generally maintaining global performance. We evaluate the impact of structure-aware noise in both diffusion and flow-matching frameworks, and show that existing isotropic models can be efficiently fine-tuned with edge-preserving noise, making our framework practical for adapting pre-trained systems. Beyond unconditional generation, our method particularly shows improvements in structure-guided tasks such as stroke-to-image synthesis, improving robustness and perceptual quality, as evidenced by consistent improvements across FID, KID, and CLIP-score.

Sascha Holl, Gurprit Singh, Hans-Peter Seidel

We present Diffusion Restore, a real-time framework for diffusion-based MCMC light transport. MCMC methods are highly suitable for sampling from complex high-dimensional distributions and for approximating integrals over them. In practice, they are often the only viable solution when direct sampling is not possible and alternative methods are either inefficient or cannot be applied due to the structure of the target distribution. However, controlling the exploration of the target distribution in MCMC methods remains challenging. Efficient exploration requires a balance between local exploration and global discovery, and local dynamics must rapidly explore individual modes without getting stuck or exhibiting excessive backtracking. The problem of global discovery has recently been addressed by the introduction of the Restore framework. In this work, we build on this framework and focus on improving local exploration. We show how to choose diffusion-based local dynamics within the Restore framework while completely avoiding Metropolis-adjustment, which is known to slow down convergence. Furthermore, we model these dynamics as nonreversible, introducing momentum in the drift and thereby enabling more directed exploration of the target distribution compared to reversible, random-walk-like dynamics. We provide a theoretical justification for the validity of our choice of local dynamics. Empirically, we demonstrate across diverse scenes that Diffusion Restore outperforms all existing MCMC light transport methods and establishes a new state of the art. In addition, we present a GPU implementation in ray tracing and compute shaders and achieve real-time frame rates. This demonstrates that Diffusion Restore is not only superior in offline rendering, but also outperforms traditional Path Tracing methods in real-time rendering settings, such as interactive applications and games.

Sascha Holl, Jente Vandersanden, Gurprit Singh et al.

Score-based generative modeling (SBGM) has achieved state-of-the-art performance in image generation, with the quality of generated images being highly dependent on the design of the forward (diffusion) process. Among these, models based on stochastic differential equations (SDEs) have proven particularly effective. While traditional methods aim to progressively destroy all image information to enable reconstruction from pure noise, we propose a class of anisotropic stochastic partial differential equations (SPDEs) that preserve the geometric structure of the data over longer time scales throughout the transformation. These SPDEs consist of a drift term that enforces deterministic destruction via structured smoothing, and a diffusion coefficient that enables random destruction through noise injection. Both components are governed by anisotropy coefficients, enabling controlled, direction-dependent information degradation. This framework provides the theoretical foundation for a novel anisotropic score-based generative model. By retaining geometric structure for longer time scales, the backward generative process can exploit residual geometric cues, leading to improved reconstruction fidelity. We empirically validate this improvement in a proof-of-concept implementation on unconditional image generation, showing that anisotropic diffusion can achieve superior image quality metrics. We demonstrate consistent improvements in both pixel and latent space experiments over the SDE-driven baseline as well as over the state-of-the-art Flow Matching approach. Finally, we demonstrate the effectiveness of the introduced anisotropy in a conditional stroke-to-image generation task.

Sascha Holl, Gurprit Singh, Hans-Peter Seidel

In light transport simulation, Markov chain Monte Carlo methods are particularly effective at exploring regions with complex lighting characteristics. However, estimator variance is a central concern across Monte Carlo methods in general. In light transport, high variance directly manifests as increased noise or, equivalently, longer rendering times at fixed image quality. Variance reduction techniques based on Rao-Blackwellization have proven particularly effective. In practice, however, the RB approach traditionally used in light transport, waste-recycling, can yield little to no measurable variance reduction, a fact we empirically confirm in this work. Motivated by this lack of effective variance reduction, we introduce a novel RB technique for the general-purpose Metropolis-Hastings algorithm that is computationally efficient and achieves substantial variance reduction. We show that this method consistently outperforms waste-recycling in terms of both variance reduction and convergence speed. Building on this result, we adapt the proposed RB approach to the recently introduced general-purpose Jump Restore algorithm, where it similarly achieves substantial variance reduction and accelerated convergence. Through extensive experiments in light transport simulation, we demonstrate that our \gls{rb} technique significantly outperforms the traditional approaches for both MH-based light transport algorithms and Jump Restore Light Transport, under both equal-time and equal-sample-count comparisons.