Morse Neural Networks for Uncertainty Quantification
This work addresses uncertainty quantification for deep learning practitioners by offering a versatile tool that unifies multiple existing techniques, though it appears incremental as it builds on prior methods like support vector machines and kernel methods.
The authors tackled the problem of uncertainty quantification in deep learning by introducing Morse neural networks, which generalize Gaussian densities to have modes on high-dimensional submanifolds, resulting in a unified model that provides a generative density, OOD detector, calibration temperature, generative sampler, and distance-aware classifier.
We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points. Fitting the Morse neural network via a KL-divergence loss yields 1) a (unnormalized) generative density, 2) an OOD detector, 3) a calibration temperature, 4) a generative sampler, along with in the supervised case 5) a distance aware-classifier. The Morse network can be used on top of a pre-trained network to bring distance-aware calibration w.r.t the training data. Because of its versatility, the Morse neural networks unifies many techniques: e.g., the Entropic Out-of-Distribution Detector of (Macêdo et al., 2021) in OOD detection, the one class Deep Support Vector Description method of (Ruff et al., 2018) in anomaly detection, or the Contrastive One Class classifier in continuous learning (Sun et al., 2021). The Morse neural network has connections to support vector machines, kernel methods, and Morse theory in topology.