Automated Text Identification Using CNN and Training Dynamics
This work addresses generalization challenges in text identification, but it is incremental as it applies existing methods to a specific dataset.
The researchers tackled the problem of improving out-of-distribution generalization in text identification by using Data Maps to analyze training dynamics on the AuTexTification dataset, finding that training a CNN only on ambiguous examples enhanced performance.
We used Data Maps to model and characterize the AuTexTification dataset. This provides insights about the behaviour of individual samples during training across epochs (training dynamics). We characterized the samples across 3 dimensions: confidence, variability and correctness. This shows the presence of 3 regions: easy-to-learn, ambiguous and hard-to-learn examples. We used a classic CNN architecture and found out that training the model only on a subset of ambiguous examples improves the model's out-of-distribution generalization.