DNF-Net: A Neural Architecture for Tabular Data
This addresses the challenge of applying neural networks to tabular data, which lacks a dominant architecture, offering a practical solution for end-to-end handling.
The paper tackles the problem of handling tabular data with deep learning by introducing DNF-Net, a neural architecture based on Boolean formulas in disjunctive normal form, and shows it significantly and consistently outperforms fully connected networks on tabular data.
A challenging open question in deep learning is how to handle tabular data. Unlike domains such as image and natural language processing, where deep architectures prevail, there is still no widely accepted neural architecture that dominates tabular data. As a step toward bridging this gap, we present DNF-Net a novel generic architecture whose inductive bias elicits models whose structure corresponds to logical Boolean formulas in disjunctive normal form (DNF) over affine soft-threshold decision terms. In addition, DNF-Net promotes localized decisions that are taken over small subsets of the features. We present an extensive empirical study showing that DNF-Nets significantly and consistently outperform FCNs over tabular data. With relatively few hyperparameters, DNF-Nets open the door to practical end-to-end handling of tabular data using neural networks. We present ablation studies, which justify the design choices of DNF-Net including the three inductive bias elements, namely, Boolean formulation, locality, and feature selection.