From pixels to notes: a computational implementation of synaesthesia for cultural artefacts
This work addresses a niche problem for researchers in computational art or psychology interested in simulating synaesthetic experiences, but it is incremental as it applies existing concepts to a specific cultural context.
The authors tackled the problem of simulating synaesthesia between music and painting by developing a deterministic Python implementation that produces melodies from colors in paintings, resulting in a computational model based on Scriabin's definition.
Synaesthesia is a condition that enables people to sense information in the form of several senses at once. This work describes a Python implementation of a simulation of synaesthesia between listening to music and viewing a painting. Based on Scriabin's definition, we developed a deterministic process to produce a melody after processing a painting, mimicking the production of notes from colours in the field of view of persons experiencing synaesthesia.