Building Language Models for Text with Named Entities
This addresses the problem of handling infrequent named entities in language models for domains like recipes and code, representing a strong specific gain.
The paper tackles the challenge of predicting named entities in text by proposing a discriminative language model that leverages entity type information, achieving 52.2% better perplexity in recipe generation and 22.06% in code generation compared to state-of-the-art models.
Text in many domains involves a significant amount of named entities. Predict- ing the entity names is often challenging for a language model as they appear less frequent on the training corpus. In this paper, we propose a novel and effective approach to building a discriminative language model which can learn the entity names by leveraging their entity type information. We also introduce two benchmark datasets based on recipes and Java programming codes, on which we evalu- ate the proposed model. Experimental re- sults show that our model achieves 52.2% better perplexity in recipe generation and 22.06% on code generation than the state-of-the-art language models.