MTCRNN: A multi-scale RNN for directed audio texture synthesis
This work provides a novel method for synthesizing complex audio textures, which could benefit sound designers and researchers working with environmental sound generation.
The paper addresses the challenge of modeling complex audio textures, such as rain and wind, which exhibit patterns across multiple timescales. It introduces a multi-scale recurrent neural network (MTCRNN) combined with a conditioning strategy to enable user-directed synthesis of these textures.
Audio textures are a subset of environmental sounds, often defined as having stable statistical characteristics within an adequately large window of time but may be unstructured locally. They include common everyday sounds such as from rain, wind, and engines. Given that these complex sounds contain patterns on multiple timescales, they are a challenge to model with traditional methods. We introduce a novel modelling approach for textures, combining recurrent neural networks trained at different levels of abstraction with a conditioning strategy that allows for user-directed synthesis. We demonstrate the model's performance on a variety of datasets, examine its performance on various metrics, and discuss some potential applications.