Block Neural Autoregressive Flow
This work addresses the computational inefficiency in normalizing flows for researchers and practitioners in machine learning, offering a more parameter-efficient method for density estimation and approximate inference, though it is incremental as it builds on existing NAF frameworks.
The paper tackled the problem of inefficient parameterization in neural autoregressive flows (NAF) for density estimation by proposing block neural autoregressive flow (B-NAF), a more compact universal approximator that uses a single feed-forward network with block matrices to ensure invertibility, resulting in competitive performance across datasets while using orders of magnitude fewer parameters.
Normalising flows (NFS) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to hand-crafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (B-NAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feed-forward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters.