Multiple Instance Learning with Trainable Decision Tree Ensembles
This work addresses MIL for small tabular data, offering an incremental improvement with a novel hybrid method.
The paper tackles the Multiple Instance Learning problem for small tabular data by proposing STE-MIL, a random forest-based model using soft decision trees and attention mechanisms, achieving results demonstrated through numerical experiments on tabular datasets.
A new random forest based model for solving the Multiple Instance Learning (MIL) problem under small tabular data, called Soft Tree Ensemble MIL (STE-MIL), is proposed. A new type of soft decision trees is considered, which is similar to the well-known soft oblique trees, but with a smaller number of trainable parameters. In order to train the trees, it is proposed to convert them into neural networks of a specific form, which approximate the tree functions. It is also proposed to aggregate the instance and bag embeddings (output vectors) by using the attention mechanism. The whole STE-MIL model, including soft decision trees, neural networks, the attention mechanism and a classifier, is trained in an end-to-end manner. Numerical experiments with tabular datasets illustrate STE-MIL. The corresponding code implementing the model is publicly available.