How DDAIR you? Disambiguated Data Augmentation for Intent Recognition
This work addresses data quality issues in intent recognition for low-resource NLP applications, but it is incremental as it builds on existing LLM-based augmentation methods.
The paper tackles the problem of ambiguous synthetic examples generated by LLMs for data augmentation in intent recognition, particularly in low-resource settings, by introducing DDAIR, which uses Sentence Transformers to detect and regenerate less ambiguous examples, showing that this approach effectively reduces ambiguity.
Large Language Models (LLMs) are effective for data augmentation in classification tasks like intent detection. In some cases, they inadvertently produce examples that are ambiguous with regard to untargeted classes. We present DDAIR (Disambiguated Data Augmentation for Intent Recognition) to mitigate this problem. We use Sentence Transformers to detect ambiguous class-guided augmented examples generated by LLMs for intent recognition in low-resource scenarios. We identify synthetic examples that are semantically more similar to another intent than to their target one. We also provide an iterative re-generation method to mitigate such ambiguities. Our findings show that sentence embeddings effectively help to (re)generate less ambiguous examples, and suggest promising potential to improve classification performance in scenarios where intents are loosely or broadly defined.