ChladniSonify: A Visual-Acoustic Mapping Method for Chladni Patterns in New Media Art Creation
For new media artists, this provides a reproducible, real-time audio-visual mapping tool for Chladni patterns, though it is an incremental application of existing methods.
ChladniSonify enables real-time mapping of Chladni patterns to sound by using a lightweight CNN with CBAM for pattern classification, achieving 99.33% accuracy and under 50 ms end-to-end latency, thus solving issues of high technical barriers, offline computation, and uncontrollable mapping in new media art.
In new media art creation, the mapping between vision and hearing is often subjective. As a classic carrier of sound visualization, Chladni patterns have great potential in building audio-visual mapping mechanisms. However, existing tools face pain points: high technical barriers for simulation, offline computing failing real-time interaction, and uncontrollable mapping rules in general sonification tools. To address these, this paper proposes ChladniSonify, a real-time visual-acoustic mapping method for Chladni patterns. Based on Kirchhoff-Love plate theory, we build a paired dataset via numerical programming and calibrate it using ANSYS finite element simulation. Focusing on the slender nodal lines of Chladni patterns, we adopt a lightweight CNN with CBAM to achieve high-precision, low-latency pattern classification. Finally, we build an end-to-end system in Python and Max/MSP, mapping recognized patterns to corresponding sine wave frequencies. Results show the system has excellent usability: the classification module achieves 99.33% accuracy on the test set with 7.03 ms inference latency; the mapped frequency matches the theoretical value with zero deviation; the average end-to-end latency is under 50 ms, meeting real-time interactive needs. This work provides a reproducible engineering prototype for Chladni audio-visual art creation.