Enabling tabular deep learning when $d \gg n$ with an auxiliary knowledge graph
This addresses a critical bottleneck in machine learning for domains with sparse data, offering a novel solution to improve model performance in high-dimensional, low-sample scenarios.
The paper tackles the problem of overfitting in tabular datasets with high-dimensional features and limited samples (d >> n) by proposing PLATO, a method that uses an auxiliary knowledge graph to regularize a multilayer perceptron, achieving up to 10.19% improvement over 13 state-of-the-art baselines across 6 datasets.
Machine learning models exhibit strong performance on datasets with abundant labeled samples. However, for tabular datasets with extremely high $d$-dimensional features but limited $n$ samples (i.e. $d \gg n$), machine learning models struggle to achieve strong performance due to the risk of overfitting. Here, our key insight is that there is often abundant, auxiliary domain information describing input features which can be structured as a heterogeneous knowledge graph (KG). We propose PLATO, a method that achieves strong performance on tabular data with $d \gg n$ by using an auxiliary KG describing input features to regularize a multilayer perceptron (MLP). In PLATO, each input feature corresponds to a node in the auxiliary KG. In the MLP's first layer, each input feature also corresponds to a weight vector. PLATO is based on the inductive bias that two input features corresponding to similar nodes in the auxiliary KG should have similar weight vectors in the MLP's first layer. PLATO captures this inductive bias by inferring the weight vector for each input feature from its corresponding node in the KG via a trainable message-passing function. Across 6 $d \gg n$ datasets, PLATO outperforms 13 state-of-the-art baselines by up to 10.19%.