Identity Curvature Laplace Approximation for Improved Out-of-Distribution Detection
This work addresses uncertainty estimation for safety-critical applications, offering an incremental improvement over traditional Bayesian methods by making Laplace approximation more efficient and effective.
The paper tackled the problem of improving out-of-distribution detection by introducing the Identity Curvature Laplace Approximation (ICLA), which enhanced performance on datasets like CIFAR-10, CIFAR-100, and ImageNet while maintaining calibration scores.
Uncertainty estimation is crucial in safety-critical applications, where robust out-of-distribution (OOD) detection is essential. Traditional Bayesian methods, though effective, are often hindered by high computational demands. As an alternative, Laplace approximation offers a more practical and efficient approach to uncertainty estimation. In this paper, we introduce the Identity Curvature Laplace Approximation (ICLA), a novel method that challenges the conventional posterior covariance formulation by using identity curvature and optimizing prior precision. This innovative design significantly enhances OOD detection performance on well-known datasets such as CIFAR-10, CIFAR-100, and ImageNet, while maintaining calibration scores. We attribute this improvement to the alignment issues between typical feature embeddings and curvature as measured by the Fisher information matrix. Our findings are further supported by demonstrating that incorporating Fisher penalty or sharpness-aware minimization techniques can greatly enhance the uncertainty estimation capabilities of standard Laplace approximation.