Why we have switched from building full-fledged taxonomies to simply detecting hypernymy relations
This work addresses evaluation challenges in NLP for researchers, but it is incremental as it builds on existing trends without introducing new methods or data.
The paper identifies issues with evaluating taxonomy learning in NLP, such as non-reproducible and costly experiments, and proposes three future directions to enhance the role of is-a relations in downstream applications.
The study of taxonomies and hypernymy relations has been extensive on the Natural Language Processing (NLP) literature. However, the evaluation of taxonomy learning approaches has been traditionally troublesome, as it mainly relies on ad-hoc experiments which are hardly reproducible and manually expensive. Partly because of this, current research has been lately focusing on the hypernymy detection task. In this paper we reflect on this trend, analyzing issues related to current evaluation procedures. Finally, we propose three potential avenues for future work so that is-a relations and resources based on them play a more important role in downstream NLP applications.