Multi-Layer Ranking with Large Language Models for News Source Recommendation
This addresses the need for reliable information sources in news consumption, though it is incremental as it builds on existing retrieval and LLM methods for a specific domain.
The paper tackled the problem of identifying trustworthy news sources by introducing an expert recommendation task based on quoted statements, using a new dataset of 23,571 quote-speaker pairs. The results showed that a multi-layer ranking framework with LLMs significantly improved both predictive and behavioral quality of the recommender system.
To seek reliable information sources for news events, we introduce a novel task of expert recommendation, which aims to identify trustworthy sources based on their previously quoted statements. To achieve this, we built a novel dataset, called NewsQuote, consisting of 23,571 quote-speaker pairs sourced from a collection of news articles. We formulate the recommendation task as the retrieval of experts based on their likelihood of being associated with a given query. We also propose a multi-layer ranking framework employing Large Language Models to improve the recommendation performance. Our results show that employing an in-context learning based LLM ranker and a multi-layer ranking-based filter significantly improve both the predictive quality and behavioural quality of the recommender system.