A Generalized Theory of Mixup for Structure-Preserving Synthetic Data
This addresses a theoretical gap in data augmentation for machine learning practitioners, though it is incremental as it builds on existing mixup techniques.
The paper tackles the problem of mixup distorting statistical properties like variance in synthetic data, and proposes a novel mixup method with a flexible weighting scheme that preserves these properties while maintaining model performance, as confirmed by numerical experiments.
Mixup is a widely adopted data augmentation technique known for enhancing the generalization of machine learning models by interpolating between data points. Despite its success and popularity, limited attention has been given to understanding the statistical properties of the synthetic data it generates. In this paper, we delve into the theoretical underpinnings of mixup, specifically its effects on the statistical structure of synthesized data. We demonstrate that while mixup improves model performance, it can distort key statistical properties such as variance, potentially leading to unintended consequences in data synthesis. To address this, we propose a novel mixup method that incorporates a generalized and flexible weighting scheme, better preserving the original data's structure. Through theoretical developments, we provide conditions under which our proposed method maintains the (co)variance and distributional properties of the original dataset. Numerical experiments confirm that the new approach not only preserves the statistical characteristics of the original data but also sustains model performance across repeated synthesis, alleviating concerns of model collapse identified in previous research.