Flexible Tails for Normalizing Flows
This addresses a bottleneck in density estimation and variational inference for machine learning applications, offering an incremental improvement over existing solutions.
The paper tackled the problem of representing heavy-tailed distributions in normalizing flows, which are limited by standard methods using heavy-tailed base distributions, and proposed a tail transform flow (TTF) that outperforms current methods, especially for high-dimensional or heavy-tailed targets.
Normalizing flows are a flexible class of probability distributions, expressed as transformations of a simple base distribution. A limitation of standard normalizing flows is representing distributions with heavy tails, which arise in applications to both density estimation and variational inference. A popular current solution to this problem is to use a heavy tailed base distribution. We argue this can lead to poor performance due to the difficulty of optimising neural networks, such as normalizing flows, under heavy tailed input. We propose an alternative, "tail transform flow" (TTF), which uses a Gaussian base distribution and a final transformation layer which can produce heavy tails. Experimental results show this approach outperforms current methods, especially when the target distribution has large dimension or tail weight.