Efficient Parallel Audio Generation using Group Masked Language Modeling
This work addresses slow inference for audio generation tasks, offering an incremental improvement over existing parallel models like SoundStorm.
The authors tackled the problem of slow inference in parallel audio generation models by proposing Group-Masked Language Modeling and Group Iterative Parallel Decoding, which enable high-quality audio synthesis with fewer iterations and outperform baselines in prompt-based audio generation.
We present a fast and high-quality codec language model for parallel audio generation. While SoundStorm, a state-of-the-art parallel audio generation model, accelerates inference speed compared to autoregressive models, it still suffers from slow inference due to iterative sampling. To resolve this problem, we propose Group-Masked Language Modeling~(G-MLM) and Group Iterative Parallel Decoding~(G-IPD) for efficient parallel audio generation. Both the training and sampling schemes enable the model to synthesize high-quality audio with a small number of iterations by effectively modeling the group-wise conditional dependencies. In addition, our model employs a cross-attention-based architecture to capture the speaker style of the prompt voice and improves computational efficiency. Experimental results demonstrate that our proposed model outperforms the baselines in prompt-based audio generation.