Unsupervised Stemming based Language Model for Telugu Broadcast News Transcription
This work addresses data insufficiency and computational challenges in ASR for Telugu broadcast news, but it is incremental as it adapts an existing method from Hindi to Telugu.
The paper tackles the problem of handling Out of Vocabulary (OOV) words in Automatic Speech Recognition (ASR) for Telugu, an Indian language with rich morphology, by proposing an unsupervised stemming method, resulting in ASR accuracy improvements of 0.76% with supervised stemming and 0.94% with unsupervised stemming.
In Indian Languages , native speakers are able to understand new words formed by either combining or modifying root words with tense and / or gender. Due to data insufficiency, Automatic Speech Recognition system (ASR) may not accommodate all the words in the language model irrespective of the size of the text corpus. It also becomes computationally challenging if the volume of the data increases exponentially due to morphological changes to the root word. In this paper a new unsupervised method is proposed for a Indian language: Telugu, based on the unsupervised method for Hindi, to generate the Out of Vocabulary (OOV) words in the language model. By using techniques like smoothing and interpolation of pre-processed data with supervised and unsupervised stemming, different issues in language model for Indian language: Telugu has been addressed. We observe that the smoothing techniques Witten-Bell and Kneser-Ney perform well when compared to other techniques on pre-processed data from supervised learning. The ASRs accuracy is improved by 0.76% and 0.94% with supervised and unsupervised stemming respectively.