Robust normalizing flows using Bernstein-type polynomials
This work is significant for researchers and practitioners using normalizing flows in domains where data perturbations are common, by offering a more robust modeling approach.
This paper addresses the challenge of perturbations in real-world data that lead to poor generalization in normalizing flows (NF) by proposing a new framework for constructing NFs using Bernstein-type polynomials. This approach aims to improve robustness against initial errors and offers theoretical advantages such as approximation error bounds and suitability for compactly supported densities.
Modeling real-world distributions can often be challenging due to sample data that are subjected to perturbations, e.g., instrumentation errors, or added random noise. Since flow models are typically nonlinear algorithms, they amplify these initial errors, leading to poor generalizations. This paper proposes a framework to construct Normalizing Flows (NF), which demonstrates higher robustness against such initial errors. To this end, we utilize Bernstein-type polynomials inspired by the optimal stability of the Bernstein basis. Further, compared to the existing NF frameworks, our method provides compelling advantages like theoretical upper bounds for the approximation error, higher interpretability, suitability for compactly supported densities, and the ability to employ higher degree polynomials without training instability. We conduct a thorough theoretical analysis and empirically demonstrate the efficacy of the proposed technique using experiments on both real-world and synthetic datasets.