Autoregressive Models: What Are They Good For?
This work challenges the effectiveness of autoregressive models for density estimation, which is a problem for researchers and practitioners in unsupervised learning and related applications.
The paper investigates autoregressive models as density estimators for tasks like image translation and outlier detection, finding that these estimates are unreliable and do not correlate with perceptual quality or help downstream tasks.
Autoregressive (AR) models have become a popular tool for unsupervised learning, achieving state-of-the-art log likelihood estimates. We investigate the use of AR models as density estimators in two settings -- as a learning signal for image translation, and as an outlier detector -- and find that these density estimates are much less reliable than previously thought. We examine the underlying optimization issues from both an empirical and theoretical perspective, and provide a toy example that illustrates the problem. Overwhelmingly, we find that density estimates do not correlate with perceptual quality and are unhelpful for downstream tasks.