Looks can be Deceptive: Distinguishing Repetition Disfluency from Reduplication
This addresses a challenge in computational linguistics for researchers and practitioners working with Indic languages, but it is incremental as it applies existing methods to a new dataset.
This paper tackled the problem of distinguishing reduplication from repetition in speech, which are linguistically distinct but similar in form, by introducing the IndicRedRep dataset for Hindi, Telugu, and Marathi and evaluating transformer-based models, achieving macro F1 scores up to 85.62% in Hindi, 83.95% in Telugu, and 84.82% in Marathi for classification.
Reduplication and repetition, though similar in form, serve distinct linguistic purposes. Reduplication is a deliberate morphological process used to express grammatical, semantic, or pragmatic nuances, while repetition is often unintentional and indicative of disfluency. This paper presents the first large-scale study of reduplication and repetition in speech using computational linguistics. We introduce IndicRedRep, a new publicly available dataset containing Hindi, Telugu, and Marathi text annotated with reduplication and repetition at the word level. We evaluate transformer-based models for multi-class reduplication and repetition token classification, utilizing the Reparandum-Interregnum-Repair structure to distinguish between the two phenomena. Our models achieve macro F1 scores of up to 85.62% in Hindi, 83.95% in Telugu, and 84.82% in Marathi for reduplication-repetition classification.