Translating Paintings Into Music Using Neural Networks
This addresses the need for an artistic tool to facilitate real-time translations between musicians and painters, though it appears incremental as it builds on existing neural network methods for cross-modal translation.
The paper tackles the problem of translating paintings into music by learning from artistic pairings of album cover art and music, with the result being a system that generates music from paintings for use in live performances or as inspiration for artists.
We propose a system that learns from artistic pairings of music and corresponding album cover art. The goal is to 'translate' paintings into music and, in further stages of development, the converse. We aim to deploy this system as an artistic tool for real time 'translations' between musicians and painters. The system's outputs serve as elements to be employed in a joint live performance of music and painting, or as generative material to be used by the artists as inspiration for their improvisation.