Auditing Algorithmic Personalization in TikTok Comment Sections
This addresses the underexplored impact of personalization algorithms on social media comment sections, particularly for users concerned about political bias and echo chambers, but it is incremental as it builds on existing auditing methods.
The study audited TikTok's comment personalization by training partisan sock-puppet accounts and found that ranking divergence in comments is greater between political groups than within them, correlating with video-level metrics like comment volume and partisan skew, though the effect is context-dependent.
Personalization algorithms are ubiquitous in modern social computing systems, yet their effects on comment sections remain underexplored. In this work, we conducted an algorithmic auditing experiment to examine comment personalization on TikTok. We trained sock-puppet accounts to exhibit left-leaning or right-leaning preferences and successfully validated 17 of them by analyzing the videos recommended on their For You Pages. We then scraped the comment sections shown to these trained partisan accounts, along with five cold-start accounts, across 65 politically neutral videos related to the 2024 U.S. presidential election that contain abundant discussions from both left-leaning and right-leaning perspectives. We find that while the composition of top comments remains largely consistent for all videos, ranking divergence between accounts from different political groups is significantly greater than that observed within the same group for some videos. This effect is strongly correlated with video-level metrics such as comment volume, engagement inequality, and partisan skew in the comment sections. Furthermore, through an exploratory case study, we find preliminary evidence that personalization can result in comment exposure aligned with an account's political leaning. However, this pattern is not universal, suggesting that the extent of politically oriented comment personalization is context-dependent.