A Siamese Deep Forest
This is an incremental improvement for machine learning practitioners working with small datasets, offering a forest-based alternative to neural networks.
The paper tackles the problem of overfitting in neural networks with limited training data by proposing Siamese Deep Forest (SDF), a modification of gcForest that serves as an alternative to Siamese neural networks, achieving competitive performance in numerical experiments.
A Siamese Deep Forest (SDF) is proposed in the paper. It is based on the Deep Forest or gcForest proposed by Zhou and Feng and can be viewed as a gcForest modification. It can be also regarded as an alternative to the well-known Siamese neural networks. The SDF uses a modified training set consisting of concatenated pairs of vectors. Moreover, it defines the class distributions in the deep forest as the weighted sum of the tree class probabilities such that the weights are determined in order to reduce distances between similar pairs and to increase them between dissimilar points. We show that the weights can be obtained by solving a quadratic optimization problem. The SDF aims to prevent overfitting which takes place in neural networks when only limited training data are available. The numerical experiments illustrate the proposed distance metric method.