Dex2HOI: Dexterous Bimanual Two-Object Interaction Generation
This work provides a method for generating more realistic and complex human-object interactions, specifically bimanual two-object manipulations, which is a crucial step for researchers and developers working on virtual agents, robotics, and animation.
This paper addresses the limitation of current 4D Human-Object Interaction (HOI) generation models, which primarily focus on single-object manipulation, by introducing Dex2HOI, a diffusion model capable of synthesizing dexterous bimanual two-object interactions from text. Dex2HOI achieves a significant inference speed-up of up to x540 over prior state-of-the-art methods while generating arbitrarily long sequences at real-time speed.
Recent advances in 4D Human-Object Interaction (HOI) generation have enabled increasingly realistic motion synthesis, particularly for single-object manipulation. Yet current research overlooks an inherent property of human behavior: people naturally coordinate both hands and manipulate multiple objects simultaneously. To address this gap, we present Dex2HOI, a unified diffusion model for single- and two-object HOI synthesis from text. At its core, Dex2HOI employs a Dual-Stream Diffusion approach, where each object is processed in a dedicated interaction stream and coordinated through bidirectional cross-attention. To synthesize the final motion, we introduce a Motion Fusion Network integrated with novel hand-relative object representations and contact-aware conditioning applied across the whole sequence. By sampling the diffusion process autoregressively over prefix-conditioned windows, Dex2HOI generates arbitrarily long sequences at real-time speed omitting redundant test-time optimization, achieving up to x540 inference speed-up over prior state-of-the-art methods. Extensive evaluation on both single- and two-object benchmarks demonstrates state-of-the-art quantitative results, marking a step beyond conventional single-object HOI generation and toward expressive multi-object manipulation. Code and models will be released upon acceptance.