Filtering Video Noise as Audio with Motion Detection to Form a Musical Instrument
This is an incremental approach for musicians or artists interested in novel audio-visual interfaces.
The paper tackled the problem of creating harmonic sounds from video signals by proposing a musical instrument that uses video for sound synthesis and note selection, with the author informally finding the resulting sounds pleasant and interesting but hard to control.
Even though they differ in the physical domain, digital video and audio share many characteristics. Both are temporal data streams often stored in buffers with 8-bit values. This paper investigates a method for creating harmonic sounds with a video signal as input. A musical instrument is proposed, that utilizes video in both a sound synthesis method, and in a controller interface for selecting musical notes at specific velocities. The resulting instrument was informally determined by the author to sound both pleasant and interesting, but hard to control, and therefore suited for synth pad sounds.