Conditioning Autoencoder Latent Spaces for Real-Time Timbre Interpolation and Synthesis
This work addresses timbre synthesis for music and audio applications, but it is incremental as it builds on existing autoencoder methods with specific optimizations.
The paper tackled the problem of generating and interpolating timbre in real-time by comparing autoencoder topologies and activation functions, finding that sigmoid activation in the bottleneck yields a more bounded and uniformly distributed embedding than leaky ReLU, and proposed a chroma conditioning vector for improved performance.
We compare standard autoencoder topologies' performances for timbre generation. We demonstrate how different activation functions used in the autoencoder's bottleneck distributes a training corpus's embedding. We show that the choice of sigmoid activation in the bottleneck produces a more bounded and uniformly distributed embedding than a leaky rectified linear unit activation. We propose a one-hot encoded chroma feature vector for use in both input augmentation and latent space conditioning. We measure the performance of these networks, and characterize the latent embeddings that arise from the use of this chroma conditioning vector. An open source, real-time timbre synthesis algorithm in Python is outlined and shared.