Regularization can make diffusion models more efficient
This addresses the high computational cost problem for users of diffusion models in generative AI, offering an incremental improvement through regularization.
The paper tackles the computational inefficiency of diffusion models by applying sparsity regularization, proving mathematically that it reduces complexity to the intrinsic data dimension and showing empirically that it yields better samples at lower cost.
Diffusion models are one of the key architectures of generative AI. Their main drawback, however, is the computational costs. This study indicates that the concept of sparsity, well known especially in statistics, can provide a pathway to more efficient diffusion pipelines. Our mathematical guarantees prove that sparsity can reduce the input dimension's influence on the computational complexity to that of a much smaller intrinsic dimension of the data. Our empirical findings confirm that inducing sparsity can indeed lead to better samples at a lower cost.