Positional Description for Numerical Normalization
This addresses critical text normalization errors in neural models for applications like Text-To-Speech and Speech Recognition, enabling more compact and data-efficient production-ready models.
The paper tackles the challenge of numerical text normalization in language models by introducing a Positional Description Scheme (PDS) that integrates placeholder value information for digit sequences, resulting in relative accuracy improvements of 23% to 51% on complex arithmetic tasks.
We present a Positional Description Scheme (PDS) tailored for digit sequences, integrating placeholder value information for each digit. Given the structural limitations of subword tokenization algorithms, language models encounter critical Text Normalization (TN) challenges when handling numerical tasks. Our schema addresses this challenge through straightforward pre-processing, preserving the model architecture while significantly simplifying number normalization, rendering the problem tractable. This simplifies the task and facilitates more compact production-ready models capable of learning from smaller datasets. Furthermore, our investigations reveal that PDS enhances the arithmetic processing capabilities of language models, resulting in a relative accuracy improvement of 23% to 51% on complex arithmetic tasks. We demonstrate that PDS effectively mitigates fatal numerical normalization errors in neural models, requiring only a modest amount of training data without rule-based Finite State Transducers (FST). We demonstrate that PDS is essential for both the Text-To-Speech and Speech Recognition text processing, enabling effective TN under production constraints.