Konrad Kowalczyk

Mateusz Barański, Jan Jasiński, Julitta Bartolewska et al.

Hallucinations of deep neural models are amongst key challenges in automatic speech recognition (ASR). In this paper, we investigate hallucinations of the Whisper ASR model induced by non-speech audio segments present during inference. By inducting hallucinations with various types of sounds, we show that there exists a set of hallucinations that appear frequently. We then study hallucinations caused by the augmentation of speech with such sounds. Finally, we describe the creation of a bag of hallucinations (BoH) that allows to remove the effect of hallucinations through the post-processing of text transcriptions. The results of our experiments show that such post-processing is capable of reducing word error rate (WER) and acts as a good safeguard against problematic hallucinations.

Piotr Masztalski, Michał Romaniuk, Jakub Żak et al.

In speaker verification, contrastive learning is gaining popularity as an alternative to the traditionally used classification-based approaches. Contrastive methods can benefit from an effective use of hard negative pairs, which are different-class samples particularly challenging for a verification model due to their similarity. In this paper, we propose CHNS - a clustering-based hard negative sampling method, dedicated for supervised contrastive speaker representation learning. Our approach clusters embeddings of similar speakers, and adjusts batch composition to obtain an optimal ratio of hard and easy negatives during contrastive loss calculation. Experimental evaluation shows that CHNS outperforms a baseline supervised contrastive approach with and without loss-based hard negative sampling, as well as a state-of-the-art classification-based approach to speaker verification by as much as 18 % relative EER and minDCF on the VoxCeleb dataset using two lightweight model architectures.

Stanisław Kacprzak, Konrad Kowalczyk

In classification tasks, the classification accuracy diminishes when the data is gathered in different domains. To address this problem, in this paper, we investigate several adversarial models for domain adaptation (DA) and their effect on the acoustic scene classification task. The studied models include several types of generative adversarial networks (GAN), with different loss functions, and the so-called cycle GAN which consists of two interconnected GAN models. The experiments are performed on the DCASE20 challenge task 1A dataset, in which we can leverage the paired examples of data recorded using different devices, i.e., the source and target domain recordings. The results of performed experiments indicate that the best performing domain adaptation can be obtained using the cycle GAN, which achieves as much as 66% relative improvement in accuracy for the target domain device, while only 6\% relative decrease in accuracy on the source domain. In addition, by utilizing the paired data examples, we are able to improve the overall accuracy over the model trained using larger unpaired data set, while decreasing the computational cost of the model training.

Szymon Woźniak, Konrad Kowalczyk

Many signal processing algorithms for distributed sensors are capable of improving their performance if the positions of sensors are known. In this paper, we focus on estimators for inferring the relative geometry of distributed arrays and sources, i.e. the setup geometry up to a scaling factor. Firstly, we present the Maximum Likelihood estimator derived under the assumption that the Direction of Arrival measurements follow the von Mises-Fisher distribution. Secondly, using unified notation, we show the relations between the cost functions of a number of state-of-the-art relative geometry estimators. Thirdly, we derive a novel estimator that exploits the concept of rays between the arrays and source event positions. Finally, we show the evaluation results for the presented estimators in various conditions, which indicate that major improvements in the probability of convergence to the optimum solution over the existing approaches can be achieved by using the proposed ray-based estimator.