Seeing Through Fog: Towards Fog-Invariant Action Recognition
For computer vision researchers, this work provides a new benchmark and method for robust action recognition in adverse weather, though the performance gains are competitive rather than breakthrough.
The paper addresses action recognition under foggy conditions, introducing FogAct, a benchmark dataset with nearly 10,000 video clips across 55 actions, and FogNet, a two-stream CLIP model that learns fog-invariant representations. FogNet achieves competitive performance with SOTA methods on FogAct and three other datasets.
Foggy conditions are commonly encountered in real-world applications; however, existing action recognition approaches typically assume favorable weather and high-quality video inputs. On foggy days, unpredictable visibility degradation and reduced contrast obstruct the extraction of semantic cues, posing significant challenges for current action recognition methods. In this paper, we mitigate the issues faced in action recognition under foggy conditions by employing two strategies. First, we present FogAct, the first benchmark dataset for foggy action recognition, consisting of paired clean and foggy videos captured with a stereo camera system. The dataset spans 10 scenes and 55 action categories, comprising nearly 10,000 video clips. Second, we propose FogNet, a two-stream CLIP model that discovers fog-invariant semantic information hidden behind the degraded videos. FogNet learns robust representations of foggy videos with guidance from clean videos, effectively capturing shared structural and motion cues between clean and foggy videos. Extensive experiments on FogAct and three other popular datasets demonstrate that our method achieves competitive performance compared with state-of-the-art (SOTA) approaches. Our FogAct and FogNet are given in our project page.