Memisis: Orchestrating and Evaluating Synthetic Data for Tabular Health Datasets
For healthcare researchers needing privacy-preserving synthetic data, Memisis provides a unified workflow with user control, but the demo results are incremental (comparing existing methods).
Memisis is a tool that orchestrates and evaluates synthetic data generation for tabular health datasets using existing synthesizers, large language models, and evaluation metrics. In a demo on a schizophrenia dataset, CTGAN, TVAE, and GaussianCopula showed comparable performance across fairness and utility metrics.
Synthetic data is widely used in healthcare to create datasets that are similar to original data but without the privacy concerns. Generating and evaluating synthetic data across privacy, utility and fairness is crucial for facilitating high quality data availability for downstream prediction tasks and clinical decision making. We present Memisis, a tool that orchestrates and evaluates synthetic data by leveraging existing synthetic data tools, the power of large language models and state-of-the-art evaluation metrics. Our tool creates a unified workflow for data generation, validation and evaluation. Users have control over the training size, training epochs and the number of synthetic rows to sample. Instead of knobs to tune synthetic data, the interactive agent allows users to specify their synthetic data generation goals and the tool will orchestrate the workflow by leveraging existing tools while performing the requisite evaluation. For the demo, we use an open source schizophrenia dataset with protected attributes related to race and gender, three different synthesizers and a local language model to orchestrate the workflow. We observe that CTGAN, TVAE and GaussianCopula have comparable performance across fairness and utility metrics. The workflow allows users flexibility and control over the data generation and evaluation process.