MambaTab: A Plug-and-Play Model for Learning Tabular Data
This provides a more efficient and accessible solution for practitioners working with tabular data across various domains, though it appears incremental as it adapts an existing SSM variant to a new data type.
The paper tackles the problem of deep learning for tabular data by introducing MambaTab, a model based on structured state-space models, which achieves superior performance with fewer parameters compared to state-of-the-art baselines on benchmark datasets.
Despite the prevalence of images and texts in machine learning, tabular data remains widely used across various domains. Existing deep learning models, such as convolutional neural networks and transformers, perform well however demand extensive preprocessing and tuning limiting accessibility and scalability. This work introduces an innovative approach based on a structured state-space model (SSM), MambaTab, for tabular data. SSMs have strong capabilities for efficiently extracting effective representations from data with long-range dependencies. MambaTab leverages Mamba, an emerging SSM variant, for end-to-end supervised learning on tables. Compared to state-of-the-art baselines, MambaTab delivers superior performance while requiring significantly fewer parameters, as empirically validated on diverse benchmark datasets. MambaTab's efficiency, scalability, generalizability, and predictive gains signify it as a lightweight, "plug-and-play" solution for diverse tabular data with promise for enabling wider practical applications.