Compositional Demographic Word Embeddings
This work addresses the challenge of personalizing language models for new users with sparse data, though it is incremental as it builds on existing embedding methods by incorporating demographic information.
The paper tackled the problem of creating personalized word embeddings for users with limited data by proposing demographic-specific word representations derived from demographic attributes like gender, age, location, and religion. The result showed that these demographic-aware embeddings outperformed generic embeddings on language modeling and word association tasks for English.
Word embeddings are usually derived from corpora containing text from many individuals, thus leading to general purpose representations rather than individually personalized representations. While personalized embeddings can be useful to improve language model performance and other language processing tasks, they can only be computed for people with a large amount of longitudinal data, which is not the case for new users. We propose a new form of personalized word embeddings that use demographic-specific word representations derived compositionally from full or partial demographic information for a user (i.e., gender, age, location, religion). We show that the resulting demographic-aware word representations outperform generic word representations on two tasks for English: language modeling and word associations. We further explore the trade-off between the number of available attributes and their relative effectiveness and discuss the ethical implications of using them.