MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning
This addresses the need for scalable and accurate approximations in large-scale ML, offering a robust method for practitioners dealing with high-dimensional data.
The paper tackles the problem of efficient approximation in large-scale machine learning by proposing a novel maximum entropy algorithm that handles hundreds of moments, showing superiority in applications like fast log determinant estimation and information-theoretic Bayesian optimization.
Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its equivalence to constrained Bayesian variational inference and demonstrate its superiority over existing approaches in two applications, namely, fast log determinant estimation and information-theoretic Bayesian optimisation.