Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Data
This addresses data scarcity for language detection tasks in specific domains like Italian job advertisements, but it is incremental as it builds on existing synthetic data methods.
The study tackled the problem of costly data collection for fine-tuning Large Language Models in non-English languages by generating synthetic data for inclusive language detection in Italian job advertisements, finding that models trained on synthetic data consistently outperformed others on both real and synthetic test datasets.
Collecting high-quality training data is essential for fine-tuning Large Language Models (LLMs). However, acquiring such data is often costly and time-consuming, especially for non-English languages such as Italian. Recently, researchers have begun to explore the use of LLMs to generate synthetic datasets as a viable alternative. This study proposes a pipeline for generating synthetic data and a comprehensive approach for investigating the factors that influence the validity of synthetic data generated by LLMs by examining how model performance is affected by metrics such as prompt strategy, text length and target position in a specific task, i.e. inclusive language detection in Italian job advertisements. Our results show that, in most cases and across different metrics, the fine-tuned models trained on synthetic data consistently outperformed other models on both real and synthetic test datasets. The study discusses the practical implications and limitations of using synthetic data for language detection tasks with LLMs.