Leveraging Acoustic and Linguistic Embeddings from Pretrained speech and language Models for Intent Classification
This work addresses the problem of limited training data and lack of linguistic information in end-to-end intent classification systems for spoken language understanding, though it appears incremental in its approach.
The paper tackles intent classification in spoken language understanding by proposing a framework that combines acoustic features from a pretrained speech recognition system and linguistic features from a pretrained language model, achieving 90.86% accuracy on ATIS and 99.07% on Fluent speech corpus.
Intent classification is a task in spoken language understanding. An intent classification system is usually implemented as a pipeline process, with a speech recognition module followed by text processing that classifies the intents. There are also studies of end-to-end system that takes acoustic features as input and classifies the intents directly. Such systems don't take advantage of relevant linguistic information, and suffer from limited training data. In this work, we propose a novel intent classification framework that employs acoustic features extracted from a pretrained speech recognition system and linguistic features learned from a pretrained language model. We use knowledge distillation technique to map the acoustic embeddings towards linguistic embeddings. We perform fusion of both acoustic and linguistic embeddings through cross-attention approach to classify intents. With the proposed method, we achieve 90.86% and 99.07% accuracy on ATIS and Fluent speech corpus, respectively.