Direct Speech to Speech Translation: A Review
It addresses the problem of real-time multilingual communication for applications like diplomacy and tourism, but is incremental as it reviews existing approaches without introducing new methods.
This review compares traditional cascade models with newer direct speech-to-speech translation (S2ST) models, highlighting that direct models reduce latency and improve naturalness by preserving vocal characteristics, but face challenges like data sparsity and high computational costs.
Speech to speech translation (S2ST) is a transformative technology that bridges global communication gaps, enabling real time multilingual interactions in diplomacy, tourism, and international trade. Our review examines the evolution of S2ST, comparing traditional cascade models which rely on automatic speech recognition (ASR), machine translation (MT), and text to speech (TTS) components with newer end to end and direct speech translation (DST) models that bypass intermediate text representations. While cascade models offer modularity and optimized components, they suffer from error propagation, increased latency, and loss of prosody. In contrast, direct S2ST models retain speaker identity, reduce latency, and improve translation naturalness by preserving vocal characteristics and prosody. However, they remain limited by data sparsity, high computational costs, and generalization challenges for low-resource languages. The current work critically evaluates these approaches, their tradeoffs, and future directions for improving real time multilingual communication.