Learning Decoupling Features Through Orthogonality Regularization
This work addresses the challenge of simultaneously extracting common and task-specific features in speech applications, which is incremental as it builds on existing models with a novel regularization technique.
The paper tackled the problem of jointly learning decoupled features for keyword spotting and speaker verification by proposing a two-branch deep network with orthogonality regularization, achieving state-of-the-art equal error rates of 1.31% on keyword spotting and 1.87% on speaker verification on the Google Speech Commands Dataset.
Keyword spotting (KWS) and speaker verification (SV) are two important tasks in speech applications. Research shows that the state-of-art KWS and SV models are trained independently using different datasets since they expect to learn distinctive acoustic features. However, humans can distinguish language content and the speaker identity simultaneously. Motivated by this, we believe it is important to explore a method that can effectively extract common features while decoupling task-specific features. Bearing this in mind, a two-branch deep network (KWS branch and SV branch) with the same network structure is developed and a novel decoupling feature learning method is proposed to push up the performance of KWS and SV simultaneously where speaker-invariant keyword representations and keyword-invariant speaker representations are expected respectively. Experiments are conducted on Google Speech Commands Dataset (GSCD). The results demonstrate that the orthogonality regularization helps the network to achieve SOTA EER of 1.31% and 1.87% on KWS and SV, respectively.