Relevance Scoring of Triples Using Ordinal Logistic Classification - The Celosia Triple Scorer at WSDM Cup 2017
This work addresses the specific challenge of triple scoring for knowledge base applications, representing an incremental improvement in a competition setting.
The authors tackled the problem of automatically predicting relevance scores for triples of type-like relations (profession and nationality) using a supervised machine learning approach, achieving an overall accuracy of 0.73 and Kendall's tau score of 0.36.
In this paper, we report our participation in the Task 2: Triple Scoring of WSDM Cup challenge 2017. In this task, we were provided with triples of "type-like" relations which were given human-annotated relevance scores ranging from 0 to 7, with 7 being the "most relevant" and 0 being the "least relevant". The task focuses on two such relations: profession and nationality. We built a system which could automatically predict the relevance scores for unseen triples. Our model is primarily a supervised machine learning based one in which we use well-designed features which are used to a make a Logistic Ordinal Regression based classification model. The proposed system achieves an overall accuracy score of 0.73 and Kendall's tau score of 0.36.