Generative Deduplication For Socia Media Data Selection
This addresses data redundancy for social media NLP pipelines, but it appears incremental as it builds on existing deduplication methods with specific enhancements.
The paper tackles the problem of severe redundancy in noisy social media data, which increases training time and model bias, by proposing a Generative Deduplication framework that removes semantically duplicate data. The results show the model reduces training samples while improving performance compared to baselines.
Social media data exhibits severe redundancy caused by its noisy nature. It leads to increased training time and model bias in its processing. To address this issue, we propose a novel Generative Deduplication framework for social media data selection by removing semantically duplicate data. While related work involves data selection in task-specific training, our model acts as an efficient pre-processing method to universally enhance social media NLP pipelines. Specifically, we train a generative model via self-supervised learning to predict a keyword to capture the semantics of noisy social media text for deduplication. Meanwhile, time-dimensional Gaussian noise is added to improve training complexity and avoid learning trivial features. Extensive experiments suggest that our model can better reduce training samples while improving performance than baselines. The results show our model's potential to broadly advance social media language understanding in effectiveness and efficiency.