Human and Automatic Speech Recognition Performance on German Oral History Interviews
This work addresses transcription accuracy for oral history archives, but it is incremental as it adapts existing methods to a specific domain.
The study measured the gap between human and machine transcription accuracy on German oral history interviews, finding a human word error rate of 8.7% and achieving machine rates of 15.6% on clean and 23.9% on noisy data after model adaptation.
Automatic speech recognition systems have accomplished remarkable improvements in transcription accuracy in recent years. On some domains, models now achieve near-human performance. However, transcription performance on oral history has not yet reached human accuracy. In the present work, we investigate how large this gap between human and machine transcription still is. For this purpose, we analyze and compare transcriptions of three humans on a new oral history data set. We estimate a human word error rate of 8.7% for recent German oral history interviews with clean acoustic conditions. For comparison with recent machine transcription accuracy, we present experiments on the adaptation of an acoustic model achieving near-human performance on broadcast speech. We investigate the influence of different adaptation data on robustness and generalization for clean and noisy oral history interviews. We optimize our acoustic models by 5 to 8% relative for this task and achieve 23.9% WER on noisy and 15.6% word error rate on clean oral history interviews.