Simple Modifications to Improve Tabular Neural Networks
This work addresses the challenge of enhancing neural networks for tabular data, which is incremental as it builds on prior models rather than introducing a new paradigm.
The paper tackled the problem of improving neural network performance on tabular data by proposing modifications to existing tabular neural network models, resulting in competitive performance with leading general-purpose models like gradient boosted decision trees.
There is growing interest in neural network architectures for tabular data. Many general-purpose tabular deep learning models have been introduced recently, with performance sometimes rivaling gradient boosted decision trees (GBDTs). These recent models draw inspiration from various sources, including GBDTs, factorization machines, and neural networks from other application domains. Previous tabular neural networks are also drawn upon, but are possibly under-considered, especially models associated with specific tabular problems. This paper focuses on several such models, and proposes modifications for improving their performance. When modified, these models are shown to be competitive with leading general-purpose tabular models, including GBDTs.