Fourier Basis Density Model
This work addresses density estimation for multi-modal distributions, which is a challenge in machine learning, but it appears incremental as it builds on existing models with specific improvements.
The authors tackled the problem of approximating multi-modal 1D densities, which are difficult to fit, by introducing a lightweight, flexible, and end-to-end trainable probability density model using a constrained Fourier basis. They achieved a lower cross entropy compared to a deep factorized model at a similar computational budget and demonstrated utility in a toy compression task.
We introduce a lightweight, flexible and end-to-end trainable probability density model parameterized by a constrained Fourier basis. We assess its performance at approximating a range of multi-modal 1D densities, which are generally difficult to fit. In comparison to the deep factorized model introduced in [1], our model achieves a lower cross entropy at a similar computational budget. In addition, we also evaluate our method on a toy compression task, demonstrating its utility in learned compression.