CTSyn: A Foundation Model for Cross Tabular Data Generation
This addresses the problem of synthetic data generation for tabular data, which is crucial for data privacy and augmentation in domains like healthcare and finance, representing a novel method rather than an incremental improvement.
The paper tackles the challenge of generating synthetic tabular data, which is difficult due to heterogeneous features, by proposing CTSyn, a diffusion-based generative foundation model that outperforms existing synthesizers on standard benchmarks in utility and diversity.
Generative Foundation Models (GFMs) have achieved remarkable success in producing high-quality synthetic data for images and text. However, their application to tabular data presents significant challenges due to the heterogeneous nature of table features. Current cross-table learning frameworks struggle because they lack a generative model backbone and an effective mechanism to decode heterogeneous feature values. To address these challenges, we propose the Cross-Table Synthesizer (CTSyn), a diffusion-based generative foundation model for tabular data generation. CTSyn comprises two key components. The first is an autoencoder network that consolidates diverse tables into a unified latent space. It dynamically reconstructs table values using a table schema embedding, allowing adaptation to heterogeneous datasets. The second is a conditional latent diffusion model that generates samples from the learned latent space, conditioned on the table schema. Through large-scale pre-training, CTSyn outperforms existing table synthesizers on standard benchmarks in both utility and diversity. These results position CTSyn as a promising framework for synthetic table generation and lay the groundwork for developing large-scale tabular foundation models.