EDGE: Editable Dance Generation From Music
This work addresses the difficulty and time-consuming nature of creating new dances for artists and creators, offering an incremental advancement with enhanced editing capabilities.
The paper tackles the problem of generating realistic and editable dances from music by introducing EDGE, a transformer-based diffusion model that achieves state-of-the-art performance, as demonstrated through quantitative metrics and a large-scale user study showing significant improvements over previous methods.
Dance is an important human art form, but creating new dances can be difficult and time-consuming. In this work, we introduce Editable Dance GEneration (EDGE), a state-of-the-art method for editable dance generation that is capable of creating realistic, physically-plausible dances while remaining faithful to the input music. EDGE uses a transformer-based diffusion model paired with Jukebox, a strong music feature extractor, and confers powerful editing capabilities well-suited to dance, including joint-wise conditioning, and in-betweening. We introduce a new metric for physical plausibility, and evaluate dance quality generated by our method extensively through (1) multiple quantitative metrics on physical plausibility, beat alignment, and diversity benchmarks, and more importantly, (2) a large-scale user study, demonstrating a significant improvement over previous state-of-the-art methods. Qualitative samples from our model can be found at our website.