Uncertainty Estimates for Ordinal Embeddings
This work provides uncertainty quantification for ordinal embeddings, which is important for researchers and practitioners in fields like psychology or machine learning dealing with noisy comparative data, but it is incremental as it builds on existing embedding methods.
The paper tackles the problem of learning from noisy triplet comparisons by embedding objects in Euclidean space, and it introduces empirical uncertainty estimates for standard embedding algorithms, showing through simulations that these estimates are well calibrated and useful for parameter selection and uncertainty quantification.
To investigate objects without a describable notion of distance, one can gather ordinal information by asking triplet comparisons of the form "Is object $x$ closer to $y$ or is $x$ closer to $z$?" In order to learn from such data, the objects are typically embedded in a Euclidean space while satisfying as many triplet comparisons as possible. In this paper, we introduce empirical uncertainty estimates for standard embedding algorithms when few noisy triplets are available, using a bootstrap and a Bayesian approach. In particular, simulations show that these estimates are well calibrated and can serve to select embedding parameters or to quantify uncertainty in scientific applications.