Perceptually Aligning Representations of Music via Noise-Augmented Autoencoders
This work addresses the challenge of aligning machine learning representations with human perception in audio processing, offering incremental improvements for applications in music analysis and brain-computer interfaces.
The paper tackles the problem of learning perceptually aligned audio representations by training autoencoders with noise-augmented encodings and perceptual losses, resulting in hierarchical structures that capture perceptually salient information more effectively than conventional methods and improve tasks like estimating surprisal in music pitches and predicting EEG-brain responses.
We argue that training autoencoders to reconstruct inputs from noised versions of their encodings, when combined with perceptual losses, yields encodings that are structured according to a perceptual hierarchy. We demonstrate the emergence of this hierarchical structure by showing that, after training an audio autoencoder in this manner, perceptually salient information is captured in coarser representation structures than with conventional training. Furthermore, we show that such perceptual hierarchies improve latent diffusion decoding in the context of estimating surprisal in music pitches and predicting EEG-brain responses to music listening. Pretrained weights are available on github.com/CPJKU/pa-audioic.