Static Visual Spatial Priors for DoA Estimation
This addresses the challenge for robots or smart voice assistants in localizing acoustic events amidst sound interference, though it is incremental as it builds on existing DoA methods by adding visual priors.
The paper tackles the problem of sound direction-of-arrival (DoA) estimation in acoustically complex environments by proposing a multi-modal approach that uses static visual spatial priors to suppress false detections. The result is a consistent improvement over classic DoA baselines, validated on a newly collected real-world dataset.
As we interact with the world, for example when we communicate with our colleagues in a large open space or meeting room, we continuously analyse the surrounding environment and, in particular, localise and recognise acoustic events. While we largely take such abilities for granted, they represent a challenging problem for current robots or smart voice assistants as they can be easily fooled by high degree of sound interference in acoustically complex environments. Preventing such failures when using solely audio data is challenging, if not impossible since the algorithms need to take into account wider context and often understand the scene on a semantic level. In this paper, we propose what to our knowledge is the first multi-modal direction of arrival (DoA) of sound, which uses static visual spatial prior providing an auxiliary information about the environment to suppress some of the false DoA detections. We validate our approach on a newly collected real-world dataset, and show that our approach consistently improves over classic DoA baselines