Gradient-Informed Training for Low-Resource Multilingual Speech Translation
For researchers working on multilingual speech translation, this work offers a principled way to mitigate negative transfer in low-resource settings.
The paper addresses representation conflicts in low-resource multilingual speech translation by proposing a gradient-informed method to determine layer-specific sharing patterns, achieving consistent improvements in translation quality across four language pairs.
In low-resource multilingual speech-to-text translation, uniform architectural sharing across languages frequently introduces representation conflicts that impede convergence. This work proposes a principled methodology to automatically determine layer-specific sharing patterns by mining training gradient information. Our approach employs three distinct analysis strategies: distance-based language clustering, self/cross-task divergence metrics for capacity allocation, and joint factorization coupled with canonical correlation analysis for subspace alignment. Extensive evaluation across four language pairs (using the SeamlessM4T-Medium architecture) demonstrates persistent improvements in translation quality metrics.