On the Robustness of Human-Object Interaction Detection against Distribution Shift
This addresses the practical applicability issue of HOI detection for real-world scenarios with distribution shifts, but it is incremental as it builds on existing methods.
The paper tackled the problem of Human-Object Interaction (HOI) detection models being insufficiently robust under distribution shifts, and proposed a cross-domain data augmentation and feature fusion strategy that significantly increased robustness across more than 40 models, with benefits on standard benchmarks.
Human-Object Interaction (HOI) detection has seen substantial advances in recent years. However, existing works focus on the standard setting with ideal images and natural distribution, far from practical scenarios with inevitable distribution shifts. This hampers the practical applicability of HOI detection. In this work, we investigate this issue by benchmarking, analyzing, and enhancing the robustness of HOI detection models under various distribution shifts. We start by proposing a novel automated approach to create the first robustness evaluation benchmark for HOI detection. Subsequently, we evaluate more than 40 existing HOI detection models on this benchmark, showing their insufficiency, analyzing the features of different frameworks, and discussing how the robustness in HOI is different from other tasks. With the insights from such analyses, we propose to improve the robustness of HOI detection methods through: (1) a cross-domain data augmentation integrated with mixup, and (2) a feature fusion strategy with frozen vision foundation models. Both are simple, plug-and-play, and applicable to various methods. Our experimental results demonstrate that the proposed approach significantly increases the robustness of various methods, with benefits on standard benchmarks, too. The dataset and code will be released.