Robin Haunschild

Mengjia Wu, Yi Zhang, Robin Haunschild et al.

Assessing the quality of scientific research is essential for scholarly communication, yet widely used approaches face limitations in scalability, subjectivity, and time delay. Recent advances in large language models (LLMs) offer new opportunities for automated research evaluation based on textual content. This study examines whether LLMs can support post-publication peer review tasks by benchmarking their outputs against expert judgments and citation-based indicators. Two evaluation tasks are constructed using articles from the H1 Connect platform: identifying high-quality articles and performing finer-grained evaluation including article rating, merit classification, and expert style commenting. Multiple model families, including BERT models, general-purpose LLMs, and reasoning oriented LLMs, are evaluated under multiple learning strategies. Results show that LLMs perform well in coarse grained evaluation tasks, achieving accuracy above 0.8 in identifying highly recommended articles. However, performance decreases substantially in fine-grained rating tasks. Few-shot prompting improves performance over zero-shot settings, while supervised fine-tuning produces the strongest and most balanced results. Retrieval augmented prompting improves classification accuracy in some cases but does not consistently strengthen alignment with citation indicators. The overall correlations between model outputs and citation indicators remain positive but moderate.

Hongkan Chen, Qingshan Zhou, Robin Haunschild et al.

In modern scientific collaboration networks, certain researchers play a pivotal role in bridging scholars who have never worked together - a phenomenon we term academic "match-makers." Despite their potential importance, the prevalence, characteristics, benefits, and long-term trajectory of these individuals remain underexplored. Using the Microsoft Academic Graph (MAG), we operationalized a match-maker as an author who, in a given publication, introduced a first-time collaboration between two co-authors, each of whom had previously collaborated with the match-maker but not with each other. We employed a configuration null model to distinguish observed patterns from random chance. Our findings reveal that the match-maker phenomenon is deliberate, prevalent, and consequential. Among authors with over 20 publications, nearly 30% have served as a match-maker, and the probability of acting as one increased eightfold from 1980 to 2019. Publications involving a match-maker are more likely to appear in high-impact journals and exhibit higher disruptiveness - particularly in larger teams - suggesting that match-makers help facilitate what we term integrative disruption. Match-makers tend to emerge early in their careers, peaking around the 20th publication and at an academic age of roughly ten years. While nearly all match-makers eventually experience "abandonment" in the sense that the connected researchers later collaborate without them, their continued involvement remains substantial and is driven by research needs rather than structural factors. This reframes abandonment not as exclusion but as a natural evolution within project-based collaborations. The academic match-maker phenomenon is a strategic feature of collaboration networks characterized by early-career emergence, context-dependent persistence, and tangible contributions to high-impact, disruptive research.

Robin Haunschild, Lutz Bornmann, Devendra Potnis et al.

One way to assess a certain aspect of the value of scientific research is to measure the attention it receives on social media. While previous research has mostly focused on the "number of mentions" of scientific research on social media, the current study applies "topic networks" to measure public attention to scientific research on Twitter. Topic networks are the networks of co-occurring author keywords in scholarly publications and networks of co-occurring hashtags in the tweets mentioning those scholarly publications. This study investigates which topics in opioid scholarly publications have received public attention on Twitter. Additionally, it investigates whether the topic networks generated from the publications tweeted by all accounts (bot and non-bot accounts) differ from those generated by non-bot accounts. Our analysis is based on a set of opioid scholarly publications from 2011 to 2019 and the tweets associated with them. We use co-occurrence network analysis to generate topic networks. Results indicated that Twitter users have mostly used generic terms to discuss opioid publications, such as "Opioid," "Pain," "Addiction," "Treatment," "Analgesics," "Abuse," "Overdose," and "Disorders." Results confirm that topic networks provide a legitimate method to visualize public discussions of health-related scholarly publications and how Twitter users discuss health-related scientific research differently from the scientific community. There was a substantial overlap between the topic networks based on the tweets by all accounts and non-bot accounts. This result indicates that it might not be necessary to exclude bot accounts for generating topic networks as they have a negligible impact on the results.

Robin Haunschild, Werner Marx

Bibliometric information retrieval in databases can employ different strategies. Com-monly, queries are performed by searching in title, abstract and/or author keywords (author vocabulary). More advanced queries employ database keywords to search in a controlled vo-cabulary. Queries based on search terms can be augmented with their citing papers if a re-search field cannot be curtailed by the search query alone. Here, we present another strategy to discover the most important papers of a research field. A marker paper is used to reveal the most important works for the relevant community. All papers co-cited with the marker paper are analyzed using reference publication year spectroscopy (RPYS). For demonstration of the marker paper approach, density functional theory (DFT) is used as a research field. Compari-sons between a prior RPYS on a publication set compiled using a keyword-based search in a controlled vocabulary and three different co-citation RPYS (RPYS-CO) analyses show very similar results. Similarities and differences are discussed.

Robin Haunschild, Sven E. Hug, Martin P. Brändle et al.

In the context of a comprehensive Microsoft Academic (MA) study, we explored in an initial step the quality of linked references data in MA in comparison with Web of Science (WoS). Linked references are the backbone of bibliometrics, because they are the basis of the times cited information in citation indexes. We found that the concordance of linked references between MA and WoS ranges from weak to non-existent for the full sample (publications of the University of Zurich with less than 50 linked references in MA). An analysis with a sample restricted to less than 50 linked references in WoS showed a strong agreement between linked references in MA and WoS.