Learning Filterbanks for End-to-End Acoustic Beamforming
This addresses the problem of improving acoustic beamforming performance for short analysis windows, which is incremental as it builds on existing hybrid neural beamforming methods.
The paper tackled the conflict between short windows for learned filterbanks in source separation and long windows for beamforming by exploring fully end-to-end hybrid neural beamforming with learned filterbanks, showing that it surpasses oracle-mask based beamforming for short windows on Clarity Challenge data.
Recent work on monaural source separation has shown that performance can be increased by using fully learned filterbanks with short windows. On the other hand it is widely known that, for conventional beamforming techniques, performance increases with long analysis windows. This applies also to most hybrid neural beamforming methods which rely on a deep neural network (DNN) to estimate the spatial covariance matrices. In this work we try to bridge the gap between these two worlds and explore fully end-to-end hybrid neural beamforming in which, instead of using the Short-Time-Fourier Transform, also the analysis and synthesis filterbanks are learnt jointly with the DNN. In detail, we explore two different types of learned filterbanks: fully learned and analytic. We perform a detailed analysis using the recent Clarity Challenge data and show that by using learnt filterbanks it is possible to surpass oracle-mask based beamforming for short windows.