Predicting Punctuation in Ancient Chinese Texts: A Multi-Layered LSTM and Attention-Based Approach
This addresses the difficulty in interpreting ancient Chinese texts for readers and scholars, but it is incremental as it extends prior work.
The paper tackles the problem of predicting punctuation in ancient Chinese texts, which lack punctuation marks, by proposing a bidirectional multi-layered LSTM with multi-head attention, and finds that this approach significantly outperforms RNNs without such components.
It was only until the 20th century when the Chinese language began using punctuation. In fact, many ancient Chinese texts contain thousands of lines with no distinct punctuation marks or delimiters in sight. The lack of punctuation in such texts makes it difficult for humans to identify when there pauses or breaks between particular phrases and understand the semantic meaning of the written text (Mogahed, 2012). As a result, unless one was educated in the ancient time period, many readers of ancient Chinese would have significantly different interpretations of the texts. We propose an approach to predict the location (and type) of punctuation in ancient Chinese texts that extends the work of Oh et al (2017) by leveraging a bidirectional multi-layered LSTM with a multi-head attention mechanism as inspired by Luong et al.'s (2015) discussion of attention-based architectures. We find that the use of multi-layered LSTMs and multi-head attention significantly outperforms RNNs that don't incorporate such components when evaluating ancient Chinese texts.