A Comparison of Word-based and Context-based Representations for Classification Problems in Health Informatics
This incremental study addresses classification problems in health informatics, such as influenza infection and drug usage detection, for researchers and practitioners in the field.
The paper compared word-based and context-based text representations for three health informatics classification tasks, finding that context-based methods like ELMo and Universal Sentence Encoder improved accuracy by 2-4% over word-based approaches.
Distributed representations of text can be used as features when training a statistical classifier. These representations may be created as a composition of word vectors or as context-based sentence vectors. We compare the two kinds of representations (word versus context) for three classification problems: influenza infection classification, drug usage classification and personal health mention classification. For statistical classifiers trained for each of these problems, context-based representations based on ELMo, Universal Sentence Encoder, Neural-Net Language Model and FLAIR are better than Word2Vec, GloVe and the two adapted using the MESH ontology. There is an improvement of 2-4% in the accuracy when these context-based representations are used instead of word-based representations.