Leonid Erlygin

Leonid Erlygin, Vladimir Zholobov, Valeriia Baklanova et al.

Machine learning models play a vital role in time series forecasting. These models, however, often overlook an important element: point uncertainty estimates. Incorporating these estimates is crucial for effective risk management, informed model selection, and decision-making.To address this issue, our research introduces a method for uncertainty estimation. We employ a surrogate Gaussian process regression model. It enhances any base regression model with reasonable uncertainty estimates. This approach stands out for its computational efficiency. It only necessitates training one supplementary surrogate and avoids any data-specific assumptions. Furthermore, this method for work requires only the presence of the base model as a black box and its respective training data. The effectiveness of our approach is supported by experimental results. Using various time-series forecasting data, we found that our surrogate model-based technique delivers significantly more accurate confidence intervals. These techniques outperform both bootstrap-based and built-in methods in a medium-data regime. This superiority holds across a range of base model types, including a linear regression, ARIMA, gradient boosting and a neural network.

Leonid Erlygin, Alexey Zaytsev

Accurate uncertainty estimation is a critical challenge in open-set recognition, where a probe biometric sample may belong to an unknown identity. It can be addressed through sample quality estimation via probabilistic embeddings. However, the low variance of probabilistic embedding only partly implies a low identification error probability: an embedding of a sample could be close to several classes in a gallery, thus yielding high uncertainty despite high sample quality. We propose HolUE - a holistic uncertainty estimation method based on a Bayesian probabilistic model; it is aware of two sources of ambiguity in the open-set recognition system: (1) the gallery uncertainty caused by overlapping classes and (2) the uncertainty of embeddings. Challenging open-set recognition datasets, such as IJB-C for the image domain and VoxBlink for the audio domain, serve as a testbed for our method. We also provide a new open-set recognition protocol for the identification of whales and dolphins. In all cases, HolUE better identifies recognition errors than alternative uncertainty estimation methods, including those based solely on sample quality.