A Character-Level Approach to the Text Normalization Problem Based on a New Causal Encoder
This work addresses a critical deployment issue in NLP by improving error detection in text normalization, though it is incremental as it builds on existing encoder-decoder frameworks.
The paper tackles the problem of undetectable 'silly errors' in text normalization by proposing a character-level attention-based encoder-decoder architecture with a new Causal Feature Extractor (CFE) encoder, achieving an initial accuracy of 92.74% and enabling automatic error detection for more robust deployment.
Text normalization is a ubiquitous process that appears as the first step of many Natural Language Processing problems. However, previous Deep Learning approaches have suffered from so-called silly errors, which are undetectable on unsupervised frameworks, making those models unsuitable for deployment. In this work, we make use of an attention-based encoder-decoder architecture that overcomes these undetectable errors by using a fine-grained character-level approach rather than a word-level one. Furthermore, our new general-purpose encoder based on causal convolutions, called Causal Feature Extractor (CFE), is introduced and compared to other common encoders. The experimental results show the feasibility of this encoder, which leverages the attention mechanisms the most and obtains better results in terms of accuracy, number of parameters and convergence time. While our method results in a slightly worse initial accuracy (92.74%), errors can be automatically detected and, thus, more readily solved, obtaining a more robust model for deployment. Furthermore, there is still plenty of room for future improvements that will push even further these advantages.