BLOOM-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancement
This addresses the need for efficient and adaptable speech enhancement systems in resource-constrained environments, though it is incremental as it builds on existing masking-based methods.
The paper tackles the problem of training scalable speech enhancement networks by proposing BLOOM-Net, a blockwise optimization method for masking-based networks, which achieves scalability with only slight performance degradation compared to end-to-end models.
In this paper, we present a blockwise optimization method for masking-based networks (BLOOM-Net) for training scalable speech enhancement networks. Here, we design our network with a residual learning scheme and train the internal separator blocks sequentially to obtain a scalable masking-based deep neural network for speech enhancement. Its scalability lets it dynamically adjust the run-time complexity depending on the test time environment. To this end, we modularize our models in that they can flexibly accommodate varying needs for enhancement performance and constraints on the resources, incurring minimal memory or training overhead due to the added scalability. Our experiments on speech enhancement demonstrate that the proposed blockwise optimization method achieves the desired scalability with only a slight performance degradation compared to corresponding models trained end-to-end.