Evaluating Generative Models for Tabular Data: Novel Metrics and Benchmarking
This work addresses a gap in evaluating generative models for tabular data, which is important for domains like network security, but it is incremental as it builds on existing metrics and focuses on specific datasets.
The paper tackled the problem of evaluating generative models for tabular data, which is challenging due to structural complexity and mixed data types, by proposing three novel metrics (FAED, FPCAD, RFIS) and benchmarking them on network intrusion detection datasets, showing that FAED effectively captures issues missed by existing metrics.
Generative models have revolutionized multiple domains, yet their application to tabular data remains underexplored. Evaluating generative models for tabular data presents unique challenges due to structural complexity, large-scale variability, and mixed data types, making it difficult to intuitively capture intricate patterns. Existing evaluation metrics offer only partial insights, lacking a comprehensive measure of generative performance. To address this limitation, we propose three novel evaluation metrics: FAED, FPCAD, and RFIS. Our extensive experimental analysis, conducted on three standard network intrusion detection datasets, compares these metrics with established evaluation methods such as Fidelity, Utility, TSTR, and TRTS. Our results demonstrate that FAED effectively captures generative modeling issues overlooked by existing metrics. While FPCAD exhibits promising performance, further refinements are necessary to enhance its reliability. Our proposed framework provides a robust and practical approach for assessing generative models in tabular data applications.