COMETA: A Corpus for Medical Entity Linking in the Social Media
This addresses the need for applications to understand public health discussions on social media, but is incremental as it primarily provides a new dataset.
The authors tackled the lack of datasets for medical entity linking in social media by introducing COMETA, a corpus of 20k expert-annotated biomedical entity mentions from Reddit linked to SNOMED CT, and found through benchmark experiments on 20 baselines that even the best techniques have a significant performance gap, with the best solution combining different data views.
Whilst there has been growing progress in Entity Linking (EL) for general language, existing datasets fail to address the complex nature of health terminology in layman's language. Meanwhile, there is a growing need for applications that can understand the public's voice in the health domain. To address this we introduce a new corpus called COMETA, consisting of 20k English biomedical entity mentions from Reddit expert-annotated with links to SNOMED CT, a widely-used medical knowledge graph. Our corpus satisfies a combination of desirable properties, from scale and coverage to diversity and quality, that to the best of our knowledge has not been met by any of the existing resources in the field. Through benchmark experiments on 20 EL baselines from string- to neural-based models we shed light on the ability of these systems to perform complex inference on entities and concepts under 2 challenging evaluation scenarios. Our experimental results on COMETA illustrate that no golden bullet exists and even the best mainstream techniques still have a significant performance gap to fill, while the best solution relies on combining different views of data.