Predictive Uncertainty through Quantization
This addresses the need for reliable confidence estimates in high-risk domains, but appears incremental as it builds on existing deep latent variable models.
The paper tackles the problem of overconfidence in deep latent variable models for high-risk domains by proposing Stochastic Quantized Activation Distributions (SQUAD), a flexible yet tractable distribution over discretized latent variables, resulting in predictive uncertainty of competitive quality.
High-risk domains require reliable confidence estimates from predictive models. Deep latent variable models provide these, but suffer from the rigid variational distributions used for tractable inference, which err on the side of overconfidence. We propose Stochastic Quantized Activation Distributions (SQUAD), which imposes a flexible yet tractable distribution over discretized latent variables. The proposed method is scalable, self-normalizing and sample efficient. We demonstrate that the model fully utilizes the flexible distribution, learns interesting non-linearities, and provides predictive uncertainty of competitive quality.