Evaluating Informal-Domain Word Representations With UrbanDictionary
This work addresses the need for better evaluation tools in informal text processing, though it appears incremental as it builds on existing representation methods with a new dataset and metric.
The paper tackles the problem of evaluating word representations for informal domains like Twitter, proposing a metric based on spelling variant proximity and a dataset collection method using UrbanDictionary to test if representations can bypass explicit text normalization.
Existing corpora for intrinsic evaluation are not targeted towards tasks in informal domains such as Twitter or news comment forums. We want to test whether a representation of informal words fulfills the promise of eliding explicit text normalization as a preprocessing step. One possible evaluation metric for such domains is the proximity of spelling variants. We propose how such a metric might be computed and how a spelling variant dataset can be collected using UrbanDictionary.