Autoencoding Random Forests
This provides a new tool for visualization, compression, clustering, and denoising in domains like tabular, image, and genomic data, but it is incremental as it builds on existing random forest and autoencoding concepts.
The authors tackled the problem of autoencoding with random forests by developing a method to learn low-dimensional embeddings and invert them back to the input space, resulting in universally consistent decoders applicable to supervised or unsupervised models.
We propose a principled method for autoencoding with random forests. Our strategy builds on foundational results from nonparametric statistics and spectral graph theory to learn a low-dimensional embedding of the model that optimally represents relationships in the data. We provide exact and approximate solutions to the decoding problem via constrained optimization, split relabeling, and nearest neighbors regression. These methods effectively invert the compression pipeline, establishing a map from the embedding space back to the input space using splits learned by the ensemble's constituent trees. The resulting decoders are universally consistent under common regularity assumptions. The procedure works with supervised or unsupervised models, providing a window into conditional or joint distributions. We demonstrate various applications of this autoencoder, including powerful new tools for visualization, compression, clustering, and denoising. Experiments illustrate the ease and utility of our method in a wide range of settings, including tabular, image, and genomic data.