Grammar-Guided Hierarchical Parsing for Long-form Audio Activity Recognition
For audio activity recognition, this method reduces the need for multi-level supervision and improves interpretability, but the improvement is incremental over prior hierarchical approaches.
This work tackles hierarchical parsing of long-form audio by inferring an Act-Sub-Event parse tree from event-level evidence using a grammar-guided decoder, achieving improved temporal-order consistency (Edit score) without requiring sub-activity or activity labels for training.
Long-form audio exhibits an inherent hierarchy: fine-grained events form sub-activities, which in turn constitute higher-level activities. Prior work often models these levels separately, leading to cross-level inconsistencies and requiring supervision at multiple levels. We formulate the problem as hierarchical parsing from event-level evidence: given detected event segments with class posteriors, we infer an order-consistent Act-Sub-Event parse tree. We propose Hierarchical Activity Grammar, encoding hierarchical composition and temporal-order constraints, and perform grammar-guided decoding that combines event evidence with a grammar prior. This yields a temporally grounded parse tree from which sub-activity segmentation and activity classification are derived, without requiring sub-activity or activity labels for training. Experiments on the long-form MultiAct audio dataset demonstrate improved temporal-order consistency (Edit score) and produces interpretable hierarchies.