Universal consistency and minimax rates for online Mondrian Forests
This provides a theoretically sound online learning method for classification and regression, extending optimal rates to high-dimensional settings.
The paper tackles the statistical inconsistency of the original Mondrian Forest algorithm by modifying it to use increasing lifetime parameters and an online updating rule, achieving consistency and the minimax rate for Lipschitz regression functions in arbitrary dimensions.
We establish the consistency of an algorithm of Mondrian Forests, a randomized classification algorithm that can be implemented online. First, we amend the original Mondrian Forest algorithm, that considers a fixed lifetime parameter. Indeed, the fact that this parameter is fixed hinders the statistical consistency of the original procedure. Our modified Mondrian Forest algorithm grows trees with increasing lifetime parameters $λ_n$, and uses an alternative updating rule, allowing to work also in an online fashion. Second, we provide a theoretical analysis establishing simple conditions for consistency. Our theoretical analysis also exhibits a surprising fact: our algorithm achieves the minimax rate (optimal rate) for the estimation of a Lipschitz regression function, which is a strong extension of previous results to an arbitrary dimension.