Richard Stern

José Novoa, Rodrigo Mahu, Alejandro Díaz et al.

This paper describes the integration of weighted delay-and-sum beamforming with speech source localization using image processing and robot head visual servoing for source tracking. We take into consideration the fact that the directivity gain provided by the beamforming depends on the angular distance between its main lobe and the main response axis of the microphone array. A visual servoing scheme is used to reduce the angular distance between the center of the video frame of a robot camera and a target object. Additionally, the beamforming strategy presented combines two information sources: the direction of the target object obtained with image processing and the audio signals provided by a microphone array. These sources of information were integrated by making use of a weighted delay-and-sum beamforming method. Experiments were carried out with a real mobile robotic testbed built with a PR2 robot. Static and dynamic robot head as well as the use of one and two external noise sources were considered. The results presented here show that the appropriate integration of visual source tracking with visual servoing and a beamforming method can lead to a reduction in WER as high as 34% compared to beamforming alone.

Juan Pablo Escudero, José Novoa, Rodrigo Mahu et al.

Reverberation and additive noise have detrimental effects on the performance of automatic speech recognition systems. In this paper we explore the ability of a DNN-based spectral feature mapping to remove the effects of reverberation and additive noise. Experiments with the CHiME-2 database show that this DNN can achieve an average reduction in WER of 4.5%, when compared to the baseline system, at SNRs equal to -6 dB, -3 dB, 0 dB and 3 dB, and just 0.8% at greater SNRs of 6 dB and 9 dB. These results suggest that this DNN is more effective in removing additive noise than reverberation. To improve the DNN performance, we combine it with the weighted prediction error (WPE) method that shows a complementary behavior. While this combination provided a reduction in WER of approximately 11% when compared with the baseline, the observed improvement is not as great as that obtained using WPE alone. However, modifications to the DNN training process were applied and an average reduction in WER equal to 18.3% was achieved when compared with the baseline system. Furthermore, the improved DNN combined with WPE achieves a reduction in WER of 7.9% when compared with WPE alone.

José Novoa, Juan Pablo Escudero, Jorge Wuth et al.

This paper evaluates the robustness of a DNN-HMM-based speech recognition system in highly-reverberant real environments using the HRRE database. The performance of locally-normalized filter bank (LNFB) and Mel filter bank (MelFB) features in combination with Non-negative Matrix Factorization (NMF), Suppression of Slowly-varying components and the Falling edge (SSF) and Weighted Prediction Error (WPE) enhancement methods are discussed and evaluated. Two training conditions were considered: clean and reverberated (Reverb). With Reverb training the use of WPE and LNFB provides WERs that are 3% and 20% lower in average than SSF and NMF, respectively. WPE and MelFB provides WERs that are 11% and 24% lower in average than SSF and NMF, respectively. With clean training, which represents a significant mismatch between testing and training conditions, LNFB features clearly outperform MelFB features. The results show that different types of training, parametrization, and enhancement techniques may work better for a specific combination of speaker-microphone distance and reverberation time. This suggests that there could be some degree of complementarity between systems trained with different enhancement and parametrization methods.

Juan Pablo Escudero, Victor Poblete, José Novoa et al.

Speech recognition in highly-reverberant real environments remains a major challenge. An evaluation dataset for this task is needed. This report describes the generation of the Highly-Reverberant Real Environment database (HRRE). This database contains 13.4 hours of data recorded in real reverberant environments and consists of 20 different testing conditions which consider a wide range of reverberation times and speaker-to-microphone distances. These evaluation sets were generated by re-recording the clean test set of the Aurora-4 database which corresponds to five loudspeaker-microphone distances in four reverberant conditions.