ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation
This addresses a specific problem in motion generation for applications like animation or robotics, but it is incremental as it builds on existing text-to-motion models.
The paper tackles the challenge of capturing intra-style diversity in stylized motion generation, where a single style should correspond to diverse motion variations, by proposing ClusterStyle, a clustering-based framework that models global and local style patterns using prototypes, and it outperforms existing state-of-the-art models in experiments.
Existing stylized motion generation models have shown their remarkable ability to understand specific style information from the style motion, and insert it into the content motion. However, capturing intra-style diversity, where a single style should correspond to diverse motion variations, remains a significant challenge. In this paper, we propose a clustering-based framework, ClusterStyle, to address this limitation. Instead of learning an unstructured embedding from each style motion, we leverage a set of prototypes to effectively model diverse style patterns across motions belonging to the same style category. We consider two types of style diversity: global-level diversity among style motions of the same category, and local-level diversity within the temporal dynamics of motion sequences. These components jointly shape two structured style embedding spaces, i.e., global and local, optimized via alignment with non-learnable prototype anchors. Furthermore, we augment the pretrained text-to-motion generation model with the Stylistic Modulation Adapter (SMA) to integrate the style features. Extensive experiments demonstrate that our approach outperforms existing state-of-the-art models in stylized motion generation and motion style transfer.