Kernelised Normalising Flows
This work addresses the need for more expressive and parameter-efficient normalising flows for density estimation and generation, particularly in applications with sparse data, though it is incremental as it builds on existing flow-based models with a kernel-based approach.
The paper tackled the problem of limited expressiveness in normalising flows due to invertibility constraints by introducing Ferumal flow, a kernelised normalising flow paradigm, which achieved competitive or superior results compared to neural network-based flows while maintaining parameter efficiency, especially excelling in low-data regimes.
Normalising Flows are non-parametric statistical models characterised by their dual capabilities of density estimation and generation. This duality requires an inherently invertible architecture. However, the requirement of invertibility imposes constraints on their expressiveness, necessitating a large number of parameters and innovative architectural designs to achieve good results. Whilst flow-based models predominantly rely on neural-network-based transformations for expressive designs, alternative transformation methods have received limited attention. In this work, we present Ferumal flow, a novel kernelised normalising flow paradigm that integrates kernels into the framework. Our results demonstrate that a kernelised flow can yield competitive or superior results compared to neural network-based flows whilst maintaining parameter efficiency. Kernelised flows excel especially in the low-data regime, enabling flexible non-parametric density estimation in applications with sparse data availability.