Improving Medical Speech-to-Text Accuracy with Vision-Language Pre-training Model
This work addresses the workload reduction for clinicians by improving transcription accuracy in the medical domain, though it is incremental as it builds on existing STT systems with a correction method.
The paper tackled the problem of low accuracy in medical speech-to-text due to insufficient datasets by proposing a vision-language pre-training model for text correction, resulting in quantitatively and clinically significant improvements in performance.
Automatic Speech Recognition (ASR) is a technology that converts spoken words into text, facilitating interaction between humans and machines. One of the most common applications of ASR is Speech-To-Text (STT) technology, which simplifies user workflows by transcribing spoken words into text. In the medical field, STT has the potential to significantly reduce the workload of clinicians who rely on typists to transcribe their voice recordings. However, developing an STT model for the medical domain is challenging due to the lack of sufficient speech and text datasets. To address this issue, we propose a medical-domain text correction method that modifies the output text of a general STT system using the Vision Language Pre-training (VLP) method. VLP combines textual and visual information to correct text based on image knowledge. Our extensive experiments demonstrate that the proposed method offers quantitatively and clinically significant improvements in STT performance in the medical field. We further show that multi-modal understanding of image and text information outperforms single-modal understanding using only text information.