RandomForestMLP: An Ensemble-Based Multi-Layer Perceptron Against Curse of Dimensionality
This work addresses overfitting issues for researchers and practitioners working with small datasets in self and semi-supervised learning, though it appears incremental as it builds on existing ensemble and neural network methods.
The authors tackled the problem of overfitting in deep learning when training on very small datasets by introducing RandomForestMLP, an ensemble-based multi-layer perceptron pipeline, and demonstrated its robustness in classification tasks on realistic image datasets.
We present a novel and practical deep learning pipeline termed RandomForestMLP. This core trainable classification engine consists of a convolutional neural network backbone followed by an ensemble-based multi-layer perceptrons core for the classification task. It is designed in the context of self and semi-supervised learning tasks to avoid overfitting while training on very small datasets. The paper details the architecture of the RandomForestMLP and present different strategies for neural network decision aggregation. Then, it assesses its robustness to overfitting when trained on realistic image datasets and compares its classification performance with existing regular classifiers.