Prosanta Barai

P. Barai, G. Leroy, P. Bisht et al.

Large Language Models (LLMs) have demonstrated immense potential in artificial intelligence across various domains, including healthcare. However, their efficacy is hindered by the need for high-quality labeled data, which is often expensive and time-consuming to create, particularly in low-resource domains like healthcare. To address these challenges, we propose a crowdsourcing (CS) framework enriched with quality control measures at the pre-, real-time-, and post-data gathering stages. Our study evaluated the effectiveness of enhancing data quality through its impact on LLMs (Bio-BERT) for predicting autism-related symptoms. The results show that real-time quality control improves data quality by 19 percent compared to pre-quality control. Fine-tuning Bio-BERT using crowdsourced data generally increased recall compared to the Bio-BERT baseline but lowered precision. Our findings highlighted the potential of crowdsourcing and quality control in resource-constrained environments and offered insights into optimizing healthcare LLMs for informed decision-making and improved patient care.

Arif Ahmed, Gondy Leroy, Stephen A. Rains et al.

Health literacy is crucial to supporting good health and is a major national goal. Audio delivery of information is becoming more popular for informing oneself. In this study, we evaluate the effect of audio enhancements in the form of information emphasis and pauses with health texts of varying difficulty and we measure health information comprehension and retention. We produced audio snippets from difficult and easy text and conducted the study on Amazon Mechanical Turk (AMT). Our findings suggest that emphasis matters for both information comprehension and retention. When there is no added pause, emphasizing significant information can lower the perceived difficulty for difficult and easy texts. Comprehension is higher (54%) with correctly placed emphasis for the difficult texts compared to not adding emphasis (50%). Adding a pause lowers perceived difficulty and can improve retention but adversely affects information comprehension.

Arif Ahmed, Gondy Leroy, Agrim Sachdeva et al.

Generative artificial intelligence is increasingly used for health information, but inaccurate outputs raise concerns about trust calibration and overreliance. This study examines whether learned dependency on generative artificial intelligence affects trust in AI-generated health information and whether visual attention cues reduce overtrust in incorrect outputs. We conducted a randomized 2 by 2 experiment with 338 participants, manipulating information accuracy and visual attention cues. Trust and dependency were measured using survey scales, and linear regression models tested main and interaction effects. Information accuracy increased trust, and learned dependency was positively associated with trust. The interaction between accuracy and dependency was significant, indicating weaker trust calibration among highly dependent users. Visual attention cues did not significantly affect trust or moderate the effect of dependency. The findings suggest that learned dependency weakens trust calibration and increases susceptibility to incorrect AI-generated health information.