First-Order Manifold Data Augmentation for Regression Learning
This addresses the need for effective data augmentation in regression tasks, which is less studied than in classification, offering a domain-independent solution that could benefit various applications, though it appears incremental as it builds on existing augmentation concepts.
The authors tackled the under-explored problem of domain-independent data augmentation for regression by introducing FOMA, a method that samples new examples from tangent planes of the train distribution, and showed it improves generalization and robustness in benchmarks, outperforming mixup-based baselines.
Data augmentation (DA) methods tailored to specific domains generate synthetic samples by applying transformations that are appropriate for the characteristics of the underlying data domain, such as rotations on images and time warping on time series data. In contrast, domain-independent approaches, e.g. mixup, are applicable to various data modalities, and as such they are general and versatile. While regularizing classification tasks via DA is a well-explored research topic, the effect of DA on regression problems received less attention. To bridge this gap, we study the problem of domain-independent augmentation for regression, and we introduce FOMA: a new data-driven domain-independent data augmentation method. Essentially, our approach samples new examples from the tangent planes of the train distribution. Augmenting data in this way aligns with the network tendency towards capturing the dominant features of its input signals. We evaluate FOMA on in-distribution generalization and out-of-distribution robustness benchmarks, and we show that it improves the generalization of several neural architectures. We also find that strong baselines based on mixup are less effective in comparison to our approach. Our code is publicly available athttps://github.com/azencot-group/FOMA.