Data-Efficient Framework for Real-world Multiple Sound Source 2D Localization
This work is significant for researchers and practitioners working on sound source localization, particularly in real-world applications where data collection is expensive and microphone array layouts can vary. It offers a method to improve model performance and generalization with less reliance on real-world labeled data.
This paper addresses the challenge of localizing multiple sound sources in real-world scenarios using deep neural networks, which typically demand extensive training data. The authors propose an adversarial learning framework that significantly improves localization performance by bridging the gap between synthetic and real data without requiring real-world labels. Additionally, they introduce an explicit transformation layer that allows the model to generalize effectively to unseen microphone array layouts during inference.
Deep neural networks have recently led to promising results for the task of multiple sound source localization. Yet, they require a lot of training data to cover a variety of acoustic conditions and microphone array layouts. One can leverage acoustic simulators to inexpensively generate labeled training data. However, models trained on synthetic data tend to perform poorly with real-world recordings due to the domain mismatch. Moreover, learning for different microphone array layouts makes the task more complicated due to the infinite number of possible layouts. We propose to use adversarial learning methods to close the gap between synthetic and real domains. Our novel ensemble-discrimination method significantly improves the localization performance without requiring any label from the real data. Furthermore, we propose a novel explicit transformation layer to be embedded in the localization architecture. It enables the model to be trained with data from specific microphone array layouts while generalizing well to unseen layouts during inference.