Regularized Maximum Likelihood for Intrinsic Dimension Estimation
This work addresses the challenge of intrinsic dimension estimation for data analysis, but it appears incremental as it builds upon existing methods with a regularization scheme.
The authors tackled the problem of estimating the intrinsic dimension of datasets by proposing a new method based on regularized maximum likelihood applied to distances between close neighbors, and they demonstrated that it outperforms two other estimators in overall performance.
We propose a new method for estimating the intrinsic dimension of a dataset by applying the principle of regularized maximum likelihood to the distances between close neighbors. We propose a regularization scheme which is motivated by divergence minimization principles. We derive the estimator by a Poisson process approximation, argue about its convergence properties and apply it to a number of simulated and real datasets. We also show it has the best overall performance compared with two other intrinsic dimension estimators.