Autoregressive Synthesis of Sparse and Semi-Structured Mixed-Type Data
This addresses a critical need for data sharing and privacy in modern systems using sparse, semi-structured data, representing a novel advancement rather than an incremental improvement.
The paper tackles the problem of generating synthetic data for sparse, semi-structured formats like JSON, which existing methods struggle with due to flattening requirements. It presents Origami, an autoregressive transformer-based architecture that outperforms baselines on fidelity, utility, and detection metrics, maintaining high privacy scores and handling datasets with up to 38% sparsity effectively.
Synthetic data generation is a critical capability for data sharing, privacy compliance, system benchmarking and test data provisioning. Existing methods assume dense, fixed-schema tabular data, yet this assumption is increasingly at odds with modern data systems - from document databases, REST APIs to data lakes - which store and exchange data in sparse, semi-structured formats like JSON. Applying existing tabular methods to such data requires flattening of nested data into wide, sparse tables which scales poorly. We present Origami, an autoregressive transformer-based architecture that tokenizes data records, including nested objects and variable length arrays, into sequences of key, value and structural tokens. This representation natively handles sparsity, mixed types and hierarchical structure without flattening or imputation. Origami outperforms baselines spanning GAN, VAE, diffusion and autoregressive architectures on fidelity, utility and detection metrics across nearly all settings, while maintaining high privacy scores. On semi-structured datasets with up to 38% sparsity, baseline synthesizers either fail to scale or degrade substantially, while Origami maintains high-fidelity synthesis that is harder to distinguish from real data. To the best of our knowledge, Origami is the first architecture capable of natively modeling and generating semi-structured data end-to-end.