Density-Regression: Efficient and Distance-Aware Deep Regressor for Uncertainty Estimation under Distribution Shifts
This addresses the efficiency issue in uncertainty estimation for regression tasks under distribution shifts, but it is incremental as it builds on existing deep regressor methods.
The paper tackles the problem of high storage and slow inference in deep ensembles for uncertainty estimation by proposing Density-Regression, which uses a density function for fast single-pass inference and proves distance awareness on feature space. Empirically, it shows competitive uncertainty estimation under distribution shifts with lower model size and faster speed, e.g., on benchmark datasets like UCI and real-world applications.
Morden deep ensembles technique achieves strong uncertainty estimation performance by going through multiple forward passes with different models. This is at the price of a high storage space and a slow speed in the inference (test) time. To address this issue, we propose Density-Regression, a method that leverages the density function in uncertainty estimation and achieves fast inference by a single forward pass. We prove it is distance aware on the feature space, which is a necessary condition for a neural network to produce high-quality uncertainty estimation under distribution shifts. Empirically, we conduct experiments on regression tasks with the cubic toy dataset, benchmark UCI, weather forecast with time series, and depth estimation under real-world shifted applications. We show that Density-Regression has competitive uncertainty estimation performance under distribution shifts with modern deep regressors while using a lower model size and a faster inference speed.