Challenges for Toxic Comment Classification: An In-Depth Error Analysis
This work addresses toxic comment classification for online moderation, but it is incremental as it builds on existing methods with an ensemble and error analysis.
The paper tackled toxic comment classification by comparing deep learning and shallow approaches on a new large dataset, proposing an ensemble that outperformed individual models, and validating results on a second dataset, enabling an extensive error analysis that revealed challenges like missing paradigmatic context and inconsistent labels.
Toxic comment classification has become an active research field with many recently proposed approaches. However, while these approaches address some of the task's challenges others still remain unsolved and directions for further research are needed. To this end, we compare different deep learning and shallow approaches on a new, large comment dataset and propose an ensemble that outperforms all individual models. Further, we validate our findings on a second dataset. The results of the ensemble enable us to perform an extensive error analysis, which reveals open challenges for state-of-the-art methods and directions towards pending future research. These challenges include missing paradigmatic context and inconsistent dataset labels.