Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detection
This addresses the challenge of acquiring natural outliers for training in OOD detection, which is crucial for reliable machine learning systems, but it is an incremental improvement over prior synthesis methods.
The paper tackled the problem of generating high-quality synthetic outliers for out-of-distribution detection by proposing the Hamiltonian Monte Carlo Outlier Synthesis framework, which achieved competitive results with state-of-the-art baselines on standard and large-scale benchmarks while being highly efficient.
Out-of-distribution (OOD) detection is crucial for developing trustworthy and reliable machine learning systems. Recent advances in training with auxiliary OOD data demonstrate efficacy in enhancing detection capabilities. Nonetheless, these methods heavily rely on acquiring a large pool of high-quality natural outliers. Some prior methods try to alleviate this problem by synthesizing virtual outliers but suffer from either poor quality or high cost due to the monotonous sampling strategy and the heavy-parameterized generative models. In this paper, we overcome all these problems by proposing the Hamiltonian Monte Carlo Outlier Synthesis (HamOS) framework, which views the synthesis process as sampling from Markov chains. Based solely on the in-distribution data, the Markov chains can extensively traverse the feature space and generate diverse and representative outliers, hence exposing the model to miscellaneous potential OOD scenarios. The Hamiltonian Monte Carlo with sampling acceptance rate almost close to 1 also makes our framework enjoy great efficiency. By empirically competing with SOTA baselines on both standard and large-scale benchmarks, we verify the efficacy and efficiency of our proposed HamOS.