Structured abbreviation expansion in context
This work addresses the challenge of normalizing abbreviated messages in informal communication, which is incremental as it builds on existing tasks like spelling correction.
The paper tackles the problem of expanding ad hoc abbreviations in informal text by creating a large open-source dataset and developing two strong baselines for abbreviation expansion.
Ad hoc abbreviations are commonly found in informal communication channels that favor shorter messages. We consider the task of reversing these abbreviations in context to recover normalized, expanded versions of abbreviated messages. The problem is related to, but distinct from, spelling correction, in that ad hoc abbreviations are intentional and may involve substantial differences from the original words. Ad hoc abbreviations are productively generated on-the-fly, so they cannot be resolved solely by dictionary lookup. We generate a large, open-source data set of ad hoc abbreviations. This data is used to study abbreviation strategies and to develop two strong baselines for abbreviation expansion