Yanjie An

Yuxiang Zhao, Yichi Zhang, Yanjie An et al.

Real-time speech-to-speech translation (S2ST) systems must balance translation quality, latency, speech naturalness, and speaker consistency. Publicly documented S2ST systems have advanced direct, multilingual, streaming, and expressive modeling, while proprietary products and APIs increasingly expose real-time translation capabilities to users. However, practical deployment remains challenging for open and reproducible systems, especially in long-form and multi-speaker conversations where partial ASR hypotheses are unstable, turn boundaries are ambiguous, and target speech must be generated with an appropriate speaker prompt. We present X-Translator, a low-cost modular cascaded S2ST system that combines streaming ASR, machine translation, and prompt-conditioned TTS through a session-level runtime controller. The system uses incremental segment commitment to convert unstable ASR streams into translation-ready units, and an online speaker prompt manager to bind source speech spans to speaker-specific voice prompts for synthesis. We evaluate translation, speech quality, and latency with OpenSTBench, compare against proprietary speech translation APIs as behavioral baselines, measure long-form voice stability, evaluate speaker preservation in multi-speaker conversations, and assess multilingual translation quality. X-Translator provides an open platform for understanding the practical trade-offs of deployment-oriented S2ST. Code and demo are available at https://github.com/zhaoyx239/X-Translator.

Yanjie An, Yuxiang Zhao, Yichi Zhang et al.

Speech translation systems increasingly span speech-to-text translation (S2TT), speech-to-speech translation (S2ST), offline translation, and streaming generation, producing outputs that differ in modality, speech realization, and timing behavior. Existing evaluation practices assess important aspects such as translation quality, speech quality, and temporal quality, but these aspects are often evaluated under separate protocols, making it difficult to compare heterogeneous systems comprehensively. To address this gap, we present OpenSTBench, a unified multidimensional evaluation framework that organizes heterogeneous speech translation outputs into a shared evaluation format. OpenSTBench supports both S2TT and S2ST systems in offline and streaming settings, and jointly evaluates translation quality, speech quality, speaker preservation, emotion and paralinguistic fidelity, temporal consistency, and latency. Through experiments on representative speech translation systems, we show that systems with strong translation quality can still differ substantially in speech quality, as well as in temporal quality. OpenSTBench provides a reproducible protocol for analyzing these cross-dimensional differences and supporting application-oriented comparison of speech translation systems. The code and datasets are available at https://github.com/sjtuayj/OpenSTBench.