Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs
This addresses the challenge of high-quality audio super-resolution for applications where paired data is scarce, though it is incremental as it builds on existing CycleGAN frameworks.
The paper tackles the problem of audio super-resolution requiring paired training data and suffering from quality degradation on out-of-domain data by proposing Dual-CycleGAN, a method that uses unpaired data and decomposes the process into domain adaptation and resampling, achieving significant performance improvements over conventional methods when paired data are unavailable.
Neural audio super-resolution models are typically trained on low- and high-resolution audio signal pairs. Although these methods achieve highly accurate super-resolution if the acoustic characteristics of the input data are similar to those of the training data, challenges remain: the models suffer from quality degradation for out-of-domain data, and paired data are required for training. To address these problems, we propose Dual-CycleGAN, a high-quality audio super-resolution method that can utilize unpaired data based on two connected cycle consistent generative adversarial networks (CycleGAN). Our method decomposes the super-resolution method into domain adaptation and resampling processes to handle acoustic mismatch in the unpaired low- and high-resolution signals. The two processes are then jointly optimized within the CycleGAN framework. Experimental results verify that the proposed method significantly outperforms conventional methods when paired data are not available. Code and audio samples are available from https://chomeyama.github.io/DualCycleGAN-Demo/.