Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer
This work addresses the problem of cross-modal translation for multiple languages, but it is incremental as it builds on existing modular methods.
The paper tackled improving speech-to-text translation by using multilingual training with modular encoders and decoders, achieving significant gains in zero-shot cross-modal transfer and outperforming a supervised XLSR-based approach for several languages.
Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation. In this work, we show that this type of approach can be further improved with multilingual training. We observe significant improvements in zero-shot cross-modal speech translation, even outperforming a supervised approach based on XLSR for several languages.