Evaluation and Explainability of Unsupervised Scholarly Collaboration Recommendations
For researchers in scholarly collaboration recommendation, this work provides a comparative evaluation of unsupervised methods and introduces a constrained evaluation setting to assess robustness beyond lexical overlap.
This paper evaluates unsupervised content-based collaboration recommendations using publication text, comparing TF-IDF, topic-based models (LDA, BERTopic), and embedding-based retrieval (SciBERT with Faiss). Results show TF-IDF performs best with full information but degrades significantly when publication overlap is reduced, while topic-based and embedding-based methods maintain more stable performance.
In this paper, we examine unsupervised, content-based collaboration recommendations using publication text in scholarly settings. We compare three families of methods: a TF-IDF baseline, topic-based models (LDA and BERTopic, including clone variants), and embedding-based retrieval using SciBERT with Faiss. To evaluate model behavior beyond simple lexical matching, we introduce a constrained setting where publication overlap between researchers is partially removed while still using historical co-authorship as proxy ground truth for post-hoc evaluation. Results show clear differences across methods. TF-IDF performs best under full information but drops significantly as overlap is reduced. In contrast, topic-based and embedding-based approaches show more stable performance, suggesting they capture broader distributional similarities, rather than relying only on direct lexical overlap. We also examine explainability through two perspectives: intrinsic topic-based explanations and post-hoc, retrieval-based explanations generated using language models. These provide complementary trade-offs between transparency and human readability.