Bidirectional Action Sequence Learning for Long-term Action Anticipation with Large Language Models
This work addresses early risk detection for automated driving and robotics, but it appears incremental as it builds on existing encoder-decoder approaches with a bidirectional twist.
The paper tackles the problem of long-term action anticipation in videos by addressing the limitations of unidirectional methods, proposing BiAnt which combines forward and backward prediction using a large language model, resulting in improved edit distance performance on the Ego4D dataset.
Video-based long-term action anticipation is crucial for early risk detection in areas such as automated driving and robotics. Conventional approaches extract features from past actions using encoders and predict future events with decoders, which limits performance due to their unidirectional nature. These methods struggle to capture semantically distinct sub-actions within a scene. The proposed method, BiAnt, addresses this limitation by combining forward prediction with backward prediction using a large language model. Experimental results on Ego4D demonstrate that BiAnt improves performance in terms of edit distance compared to baseline methods.