Learning Conceptual-Contextual Embeddings for Medical Text
This addresses the need for better medical text processing by integrating external knowledge, though it appears incremental as an adaptation of existing embedding methods to a specific domain.
The paper tackled the problem of incorporating structured knowledge into text representations for medical NLP by introducing Conceptual-Contextual (CC) embeddings, which encode a knowledge graph into a context model and showed a major performance boost on supervised tasks using EHRs and benchmarks.
External knowledge is often useful for natural language understanding tasks. We introduce a contextual text representation model called Conceptual-Contextual (CC) embeddings, which incorporates structured knowledge into text representations. Unlike entity embedding methods, our approach encodes a knowledge graph into a context model. CC embeddings can be easily reused for a wide range of tasks just like pre-trained language models. Our model effectively encodes the huge UMLS database by leveraging semantic generalizability. Experiments on electronic health records (EHRs) and medical text processing benchmarks showed our model gives a major boost to the performance of supervised medical NLP tasks.