Dance2Music: Automatic Dance-driven Music Generation
This work addresses a novel challenge in computational creativity for creative expression and entertainment applications, though it appears to be an early exploration.
The paper tackles the problem of automatically generating music for a given dance video, which is under-explored compared to generating dance from music, and presents both offline and online approaches with results evaluated via human studies.
Dance and music typically go hand in hand. The complexities in dance, music, and their synchronisation make them fascinating to study from a computational creativity perspective. While several works have looked at generating dance for a given music, automatically generating music for a given dance remains under-explored. This capability could have several creative expression and entertainment applications. We present some early explorations in this direction. We present a search-based offline approach that generates music after processing the entire dance video and an online approach that uses a deep neural network to generate music on-the-fly as the video proceeds. We compare these approaches to a strong heuristic baseline via human studies and present our findings. We have integrated our online approach in a live demo! A video of the demo can be found here: https://sites.google.com/view/dance2music/live-demo.