"How trustworthy is this research?" Designing a Tool to Help Readers Understand Evidence and Uncertainty in Science Journalism
This addresses the problem of misinformation and pseudoscience for general readers of science journalism, but it is incremental as it builds on existing design approaches.
The study designed and evaluated a tool called the Scientific Evidence Indicator to help readers of science journalism assess the strength and uncertainty of scientific evidence in health news, showing some success in helping readers recognize peer-reviewed studies but facing challenges in deeper understanding.
This article reports on a Research through Design study exploring how to design a tool for helping readers of science journalism understand the strength and uncertainty of scientific evidence in news stories about health science, using both textual and visual information. A central aim has been to teach readers about criteria for assessing scientific evidence, in particular in order to help readers differentiate between science and pseudoscience. Working in a research-in-the-wild collaboration with a website for popular science, the study presents the design and evaluation of the Scientific Evidence Indicator, which uses metadata about scientific publications to present an assessment of evidence strength to the readers. Evaluations of the design demonstrate some success in helping readers recognize whether studies have undergone scientific peer review or not, but point to challenges in facilitating a more in-depth understanding. Insights from the study point to a potential for developing similar tools aimed at journalists rather than directly at audiences.