LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence
This work addresses the need for foundation models in structured data to complement language and physical-world models for general intelligence, offering a novel approach with broad applicability across tabular tasks.
The authors tackled the problem of general intelligence by developing large structured-data models (LimiX-16M and LimiX-2M) that handle tabular tasks through query-based conditional prediction, achieving superior performance across 11 benchmarks in classification, regression, missing value imputation, and data generation, often by substantial margins, while avoiding task-specific training.
We argue that progress toward general intelligence requires complementary foundation models grounded in language, the physical world, and structured data. This report presents LimiX-16M and LimiX-2M, two instantiations of our large structured-data models (LDMs). Both models treat structured data as a joint distribution over variables and missingness, thus capable of addressing a wide range of tabular tasks through query-based conditional prediction via a single model. They are pretrained using masked joint-distribution modeling with an episodic, context-conditional objective, supporting rapid, training-free adaptation at inference. We evaluate LimiX models across 11 large structured-data benchmarks with broad regimes of sample size, feature dimensionality, class number, categorical-to-numerical feature ratio, missingness, and sample-to-feature ratios. LimiX-16M consistently surpasses strong baselines, as shown in Figure 1 and Figure 2. The superiority holds across a wide range of tasks, such as classification, regression, missing value imputation, and data generation, often by substantial margins, while avoiding task-specific architectures or bespoke training per task. Notably, LimiX-2M delivers strong results under tight compute and memory budgets. We also present the first scaling law study for LDMs, revealing how data and model scaling jointly influence downstream performance and offering quantitative guidance for tabular foundation modeling. All LimiX models are publicly accessible under Apache 2.0.