Pre-training for low resource speech-to-intent applications
This work addresses the problem of reducing training data requirements for low-resource speech-to-intent applications, which is incremental as it builds on prior NMF and capsule network methods.
The paper tackles the challenge of training data scarcity in user-taught speech-to-intent systems by combining an ASR encoder with existing NMF/capsule network decoders and using pre-training. It shows that the pre-trained ASR-NMF framework significantly outperforms other models, though limitations are noted for different command-and-control applications.
Designing a speech-to-intent (S2I) agent which maps the users' spoken commands to the agents' desired task actions can be challenging due to the diverse grammatical and lexical preference of different users. As a remedy, we discuss a user-taught S2I system in this paper. The user-taught system learns from scratch from the users' spoken input with action demonstration, which ensure it is fully matched to the users' way of formulating intents and their articulation habits. The main issue is the scarce training data due to the user effort involved. Existing state-of-art approaches in this setting are based on non-negative matrix factorization (NMF) and capsule networks. In this paper we combine the encoder of an end-to-end ASR system with the prior NMF/capsule network-based user-taught decoder, and investigate whether pre-training methodology can reduce training data requirements for the NMF and capsule network. Experimental results show the pre-trained ASR-NMF framework significantly outperforms other models, and also, we discuss limitations of pre-training with different types of command-and-control(C&C) applications.