Privately Fine-Tuned LLMs Preserve Temporal Dynamics in Tabular Data
This addresses the challenge of preserving temporal dynamics in sensitive longitudinal data like electronic health records, offering a novel solution for practitioners in healthcare and other domains with sequential data.
The paper tackled the problem of generating differentially private synthetic tabular data for longitudinal datasets, where traditional methods fail to preserve temporal coherence, and introduced PATH, a framework using privately fine-tuned LLMs that reduced distributional distance to real trajectories by over 60% and state transition errors by nearly 50% compared to leading methods.
Research on differentially private synthetic tabular data has largely focused on independent and identically distributed rows where each record corresponds to a unique individual. This perspective neglects the temporal complexity in longitudinal datasets, such as electronic health records, where a user contributes an entire (sub) table of sequential events. While practitioners might attempt to model such data by flattening user histories into high-dimensional vectors for use with standard marginal-based mechanisms, we demonstrate that this strategy is insufficient. Flattening fails to preserve temporal coherence even when it maintains valid marginal distributions. We introduce PATH, a novel generative framework that treats the full table as the unit of synthesis and leverages the autoregressive capabilities of privately fine-tuned large language models. Extensive evaluations show that PATH effectively captures long-range dependencies that traditional methods miss. Empirically, our method reduces the distributional distance to real trajectories by over 60% and reduces state transition errors by nearly 50% compared to leading marginal mechanisms while achieving similar marginal fidelity.