An Unsupervised Probability Model for Speech-to-Translation Alignment of Low-Resource Languages
This addresses the challenge of utilizing translated speech data for documenting endangered languages or training speech translation systems, though it is incremental as it builds on existing methods.
The paper tackles the problem of automatically aligning spoken words with translations in low-resource languages, achieving significantly better performance than neural and strong baselines in an extremely low-resource scenario.
For many low-resource languages, spoken language resources are more likely to be annotated with translations than with transcriptions. Translated speech data is potentially valuable for documenting endangered languages or for training speech translation systems. A first step towards making use of such data would be to automatically align spoken words with their translations. We present a model that combines Dyer et al.'s reparameterization of IBM Model 2 (fast-align) and k-means clustering using Dynamic Time Warping as a distance metric. The two components are trained jointly using expectation-maximization. In an extremely low-resource scenario, our model performs significantly better than both a neural model and a strong baseline.