REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers
This addresses the lack of models for generating realistic relational datasets, which is important for data privacy and augmentation in fields like healthcare or finance, though it appears incremental as it builds on existing transformer methods.
The paper tackles the problem of generating synthetic relational tabular data, which is challenging due to the need to model parent tables and their relationships, by introducing REaLTabFormer, a model that uses GPT-2 and Seq2Seq components to produce realistic datasets, achieving state-of-the-art results on prediction tasks for large non-relational datasets without fine-tuning.
Tabular data is a common form of organizing data. Multiple models are available to generate synthetic tabular datasets where observations are independent, but few have the ability to produce relational datasets. Modeling relational data is challenging as it requires modeling both a "parent" table and its relationships across tables. We introduce REaLTabFormer (Realistic Relational and Tabular Transformer), a tabular and relational synthetic data generation model. It first creates a parent table using an autoregressive GPT-2 model, then generates the relational dataset conditioned on the parent table using a sequence-to-sequence (Seq2Seq) model. We implement target masking to prevent data copying and propose the $Q_δ$ statistic and statistical bootstrapping to detect overfitting. Experiments using real-world datasets show that REaLTabFormer captures the relational structure better than a baseline model. REaLTabFormer also achieves state-of-the-art results on prediction tasks, "out-of-the-box", for large non-relational datasets without needing fine-tuning.