Action Sensitivity Learning for the Ego4D Episodic Memory Challenge 2023
This work addresses video understanding challenges for episodic memory tasks, representing an incremental improvement with strong performance gains.
The paper tackled the Ego4D Episodic Memory Benchmark by proposing an Action Sensitivity Learning framework to capture frame discrepancies, achieving first place in Moment Queries with 29.34 mAP and second in Natural Language Queries with 19.79 mean R1.
This report presents ReLER submission to two tracks in the Ego4D Episodic Memory Benchmark in CVPR 2023, including Natural Language Queries and Moment Queries. This solution inherits from our proposed Action Sensitivity Learning framework (ASL) to better capture discrepant information of frames. Further, we incorporate a series of stronger video features and fusion strategies. Our method achieves an average mAP of 29.34, ranking 1st in Moment Queries Challenge, and garners 19.79 mean R1, ranking 2nd in Natural Language Queries Challenge. Our code will be released.