Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026
For researchers in multilingual speech instruction following, this work provides a practical data augmentation method and a decoding strategy that improves performance on semantic tasks, though the approach is incremental.
KIT's submission to the IWSLT 2026 Instruction Following Track uses a data augmentation pipeline to convert short-form corpora into long-form training data, generating over 1M instances across six tasks and four languages. They identify that likelihood-based re-ranking degrades semantic tasks and resolve this by combining it with Minimum Bayes Risk decoding.
With the advent of Large Language Models, single-task and token-based multi-task models have evolved into instruction-based systems that infer task and target language implicitly from natural language prompts. This trend is reflected in IWSLT's Instruction Following Track, which this year introduced new tasks including an unknown surprise task, posing a genuine challenge against overfitting to known tasks. We present KIT's submission to the Long and Short Instruction Following tracks in the unconstrained setting. Our approach combines a general data augmentation pipeline that converts short-form corpora into long-form training data through segment concatenation, LLM-based label generation, and cross-lingual translation, yielding over 1M instances across six tasks and four languages. We further show that likelihood-based re-ranking, while highly effective for ASR, systematically degrades semantic tasks by spuriously selecting candidates generated from segmented audio processing rather than holistic long-form inference, a failure mode resolved by combining likelihood with Minimum Bayes Risk decoding.