HAKE: A Knowledge Engine Foundation for Human Activity Understanding
This work addresses a fundamental problem in AI for applications like health care and behavior analysis, offering a novel approach to improve activity recognition.
The paper tackles the challenge of human activity understanding by proposing a two-stage paradigm that maps pixels to atomic activity primitives and then uses interpretable logic rules to infer semantics, achieving superior generalization and performance on benchmarks.
Human activity understanding is of widespread interest in artificial intelligence and spans diverse applications like health care and behavior analysis. Although there have been advances in deep learning, it remains challenging. The object recognition-like solutions usually try to map pixels to semantics directly, but activity patterns are much different from object patterns, thus hindering success. In this work, we propose a novel paradigm to reformulate this task in two stages: first mapping pixels to an intermediate space spanned by atomic activity primitives, then programming detected primitives with interpretable logic rules to infer semantics. To afford a representative primitive space, we build a knowledge base including 26+ M primitive labels and logic rules from human priors or automatic discovering. Our framework, the Human Activity Knowledge Engine (HAKE), exhibits superior generalization ability and performance upon canonical methods on challenging benchmarks. Code and data are available at http://hake-mvig.cn/.