Evaluation Metrics for Text Data Augmentation in NLP
This work tackles the problem of inconsistent method comparisons in text data augmentation for researchers and practitioners, but it is incremental as it focuses on organizing existing metrics rather than introducing new ones.
The paper addresses the lack of standardized evaluation metrics for text data augmentation in NLP, proposing a taxonomy to organize categories for implementation tools and metrics calculation to guide unified benchmarking.
Recent surveys on data augmentation for natural language processing have reported different techniques and advancements in the field. Several frameworks, tools, and repositories promote the implementation of text data augmentation pipelines. However, a lack of evaluation criteria and standards for method comparison due to different tasks, metrics, datasets, architectures, and experimental settings makes comparisons meaningless. Also, a lack of methods unification exists and text data augmentation research would benefit from unified metrics to compare different augmentation methods. Thus, academics and the industry endeavor relevant evaluation metrics for text data augmentation techniques. The contribution of this work is to provide a taxonomy of evaluation metrics for text augmentation methods and serve as a direction for a unified benchmark. The proposed taxonomy organizes categories that include tools for implementation and metrics calculation. Finally, with this study, we intend to present opportunities to explore the unification and standardization of text data augmentation metrics.