KMLP: A Scalable Hybrid Architecture for Web-Scale Tabular Data Modeling
This work provides a scalable deep learning solution for practitioners working with large-scale web tabular data, addressing the limitations of existing models like GBDTs on anisotropic, heavy-tailed, and non-stationary features.
This paper addresses the challenge of predictive modeling on web-scale tabular data, which often contains billions of instances and hundreds of heterogeneous numerical features. The authors introduce KMLP, a hybrid deep learning architecture that combines a shallow Kolmogorov-Arnold Network (KAN) front-end with a Gated Multilayer Perceptron (gMLP) backbone. KMLP achieves state-of-the-art performance on public benchmarks and an industrial dataset, demonstrating improved advantages over baselines like GBDTs, especially at larger scales.
Predictive modeling on web-scale tabular data with billions of instances and hundreds of heterogeneous numerical features faces significant scalability challenges. These features exhibit anisotropy, heavy-tailed distributions, and non-stationarity, creating bottlenecks for models like Gradient Boosting Decision Trees and requiring laborious manual feature engineering. We introduce KMLP, a hybrid deep architecture integrating a shallow Kolmogorov-Arnold Network (KAN) front-end with a Gated Multilayer Perceptron (gMLP) backbone. The KAN front-end uses learnable activation functions to automatically model complex non-linear transformations for each feature, while the gMLP backbone captures high-order interactions. Experiments on public benchmarks and an industrial dataset with billions of samples show KMLP achieves state-of-the-art performance, with advantages over baselines like GBDTs increasing at larger scales, validating KMLP as a scalable deep learning paradigm for large-scale web tabular data.