Deep factorization for speech signal
This work addresses speech processing tasks like speaker and emotion recognition, but appears incremental as it builds on prior deep learning methods for factorization.
The paper tackles the problem of factorizing speech signals into individual informative factors, which is difficult due to mixed factors, and presents a cascade deep factorization framework that infers factors sequentially, enabling high-accuracy recovery of the original speech spectrum.
Various informative factors mixed in speech signals, leading to great difficulty when decoding any of the factors. An intuitive idea is to factorize each speech frame into individual informative factors, though it turns out to be highly difficult. Recently, we found that speaker traits, which were assumed to be long-term distributional properties, are actually short-time patterns, and can be learned by a carefully designed deep neural network (DNN). This discovery motivated a cascade deep factorization (CDF) framework that will be presented in this paper. The proposed framework infers speech factors in a sequential way, where factors previously inferred are used as conditional variables when inferring other factors. We will show that this approach can effectively factorize speech signals, and using these factors, the original speech spectrum can be recovered with a high accuracy. This factorization and reconstruction approach provides potential values for many speech processing tasks, e.g., speaker recognition and emotion recognition, as will be demonstrated in the paper.