Hyperbolic Deep Learning for Chinese Natural Language Understanding
This addresses the challenge of Chinese NLP by enabling character-based models to surpass word-based ones, potentially reducing dependency on segmentation tasks.
The paper tackled the problem of Chinese natural language understanding by using hyperbolic geometry to model hierarchical relationships, resulting in a character-based Transformer that outperformed its word-based Euclidean equivalent, circumventing the need for Chinese Word Segmentation.
Recently hyperbolic geometry has proven to be effective in building embeddings that encode hierarchical and entailment information. This makes it particularly suited to modelling the complex asymmetrical relationships between Chinese characters and words. In this paper we first train a large scale hyperboloid skip-gram model on a Chinese corpus, then apply the character embeddings to a downstream hyperbolic Transformer model derived from the principles of gyrovector space for Poincare disk model. In our experiments the character-based Transformer outperformed its word-based Euclidean equivalent. To the best of our knowledge, this is the first time in Chinese NLP that a character-based model outperformed its word-based counterpart, allowing the circumvention of the challenging and domain-dependent task of Chinese Word Segmentation (CWS).