Pareto Optimal Re-ranking with Semi-Automated Content Credibility Detection
For social media platforms and users, this provides a practical way to balance content ranking fidelity with credibility, though the improvement is incremental over existing ranking methods.
The paper presents an optimization-based method to improve credibility of social media feeds by refining rankings via dual-objective optimization (minimizing Spearman's footrule distance and maximizing expected credibility), achieving at most 7% deviation from the Pareto optimal front on real-world X (Twitter) data.
Social media posts often include misinformative or misleading content, diminishing the expected credibility of content feeds. We present an optimization-based method to improve the credibility of news content on social media feeds by refining existing content rankings. This method is based on a dual-objective optimization approach that minimizes the Spearman's footrule distance to the original ranking to maintain the original content order while incorporating an additional linear cost objective to elevate the expected credibility of the content feed. Additionally, we propose a robust semi-automated pipeline for assigning credibility scores to content based on a mixture of retrieval-augmented score assignments and human-generated fact-checks. This semi-automated pipeline helps ground the credibility assignment using human-generated labels while ensuring the algorithm extends to posts with few or no human-generated labels. We showcase our approach through an experimental setup using real-world data collected over X (Twitter), where we assign the credibility scores based on a mixture of user-generated community notes and retrieval augmented generation. The method we present leads to at most 7% deviation in both optimization objectives from the Pareto optimal front with known initial ranking values. Additionally, the algorithm allows for incorporating different measures for source credibility, making it applicable across various social media platforms.