Learning Linearity in Audio Consistency Autoencoders via Implicit Regularization
This work addresses the need for structured latent spaces in audio processing, offering a straightforward technique for more intuitive and efficient manipulation, though it is incremental as it builds on existing autoencoder frameworks without architectural changes.
The paper tackled the problem of non-linear latent spaces in audio autoencoders preventing intuitive algebraic manipulation by introducing a training methodology using data augmentation to induce linearity in Consistency Autoencoders, resulting in preserved reconstruction fidelity and enabling practical applications like music source composition and separation via latent arithmetic.
Audio autoencoders learn useful, compressed audio representations, but their non-linear latent spaces prevent intuitive algebraic manipulation such as mixing or scaling. We introduce a simple training methodology to induce linearity in a high-compression Consistency Autoencoder (CAE) by using data augmentation, thereby inducing homogeneity (equivariance to scalar gain) and additivity (the decoder preserves addition) without altering the model's architecture or loss function. When trained with our method, the CAE exhibits linear behavior in both the encoder and decoder while preserving reconstruction fidelity. We test the practical utility of our learned space on music source composition and separation via simple latent arithmetic. This work presents a straightforward technique for constructing structured latent spaces, enabling more intuitive and efficient audio processing.