A Pranav

Pranav A, Isabelle Augenstein

Simplified Chinese to Traditional Chinese character conversion is a common preprocessing step in Chinese NLP. Despite this, current approaches have poor performance because they do not take into account that a simplified Chinese character can correspond to multiple traditional characters. Here, we propose a model that can disambiguate between mappings and convert between the two scripts. The model is based on subword segmentation, two language models, as well as a method for mapping between subword sequences. We further construct benchmark datasets for topic classification and script conversion. Our proposed method outperforms previous Chinese Character conversion approaches by 6 points in accuracy. These results are further confirmed in a downstream application, where 2kenize is used to convert pretraining dataset for topic classification. An error analysis reveals that our method's particular strengths are in dealing with code-mixing and named entities.

Pranav A, Nick Sukiennik, Pan Hui

The work herein describes a system for automatic news category and keyphrase labeling, presented in the context of our motivation to improve the speed at which a user can find relevant and interesting content within an aggregation platform. A set of 12 discrete categories were applied to over 500,000 news articles for training a neural network, to be used to facilitate the more in-depth task of extracting the most significant keyphrases. The latter was done using three methods: statistical, graphical and numerical, using the pre-identified category label to improve relevance of extracted phrases. The results are presented in a demo in which the articles are pre-populated via News API, and upon being selected, the category and keyphrase labels will be computed via the methods explained herein.