Fluent Translations from Disfluent Speech in End-to-End Speech Translation
This addresses the challenge of disfluencies in speech translation for conversational applications, but it is incremental as it adapts existing models to a new task.
The paper tackles the problem of translating disfluent conversational speech to fluent text by using an end-to-end sequence-to-sequence model on the Fisher Spanish-English dataset, directly generating translations with disfluencies removed and establishing a baseline for this task.
Spoken language translation applications for speech suffer due to conversational speech phenomena, particularly the presence of disfluencies. With the rise of end-to-end speech translation models, processing steps such as disfluency removal that were previously an intermediate step between speech recognition and machine translation need to be incorporated into model architectures. We use a sequence-to-sequence model to translate from noisy, disfluent speech to fluent text with disfluencies removed using the recently collected `copy-edited' references for the Fisher Spanish-English dataset. We are able to directly generate fluent translations and introduce considerations about how to evaluate success on this task. This work provides a baseline for a new task, the translation of conversational speech with joint removal of disfluencies.