Autoencoder Trees
This work addresses the need for interpretable and hierarchical feature learning in autoencoders, but it is incremental as it adapts existing tree methods to the autoencoder framework.
The paper tackles the problem of learning hierarchical representations in autoencoders by implementing encoding and decoding functions with decision trees, specifically soft decision trees with multivariate splits, and reports that on handwritten digit and news data, it yields good reconstruction error compared to traditional autoencoder perceptrons.
We discuss an autoencoder model in which the encoding and decoding functions are implemented by decision trees. We use the soft decision tree where internal nodes realize soft multivariate splits given by a gating function and the overall output is the average of all leaves weighted by the gating values on their path. The encoder tree takes the input and generates a lower dimensional representation in the leaves and the decoder tree takes this and reconstructs the original input. Exploiting the continuity of the trees, autoencoder trees are trained with stochastic gradient descent. On handwritten digit and news data, we see that the autoencoder trees yield good reconstruction error compared to traditional autoencoder perceptrons. We also see that the autoencoder tree captures hierarchical representations at different granularities of the data on its different levels and the leaves capture the localities in the input space.