Room-acoustic simulations as an alternative to measurements for audio-algorithm evaluation
This addresses the need for more efficient and diverse evaluation methods in audio signal processing and machine learning, though it is incremental as it compares existing simulation types rather than introducing a new paradigm.
The paper tackled the problem of limited and costly real-world acoustic measurements for evaluating audio algorithms by exploring room-acoustic simulations as an alternative, finding that numerical wave-based simulations matched measurement results for three algorithms while geometrical acoustic simulations did not.
Audio-signal-processing and audio-machine-learning (ASP/AML) algorithms are ubiquitous in modern technology like smart devices, wearables, and entertainment systems. Development of such algorithms and models typically involves a formal evaluation to demonstrate their effectiveness and progress beyond the state-of-the-art. Ideally, a thorough evaluation should cover many diverse application scenarios and room-acoustic conditions. However, in practice, evaluation datasets are often limited in size and diversity because they rely on costly and time-consuming measurements. This paper explores how room-acoustic simulations can be used for evaluating ASP/AML algorithms. To this end, we evaluate three ASP/AML algorithms with room-acoustic measurements and data from different simulation engines, and assess the match between the evaluation results obtained from measurements and simulations. The presented investigation compares a numerical wave-based solver with two geometrical acoustics simulators. While numerical wave-based simulations yielded similar evaluation results as measurements for all three evaluated ASP/AML algorithms, geometrical acoustic simulations could not replicate the measured evaluation results as reliably.