Lost in Transmission: On the Impact of Networking Corruptions on Video Machine Learning Models
This work addresses the problem of networking corruptions in video datasets for machine learning researchers, but it is incremental as it builds on existing robustness studies without introducing new solutions.
The study investigated how networking corruptions, such as smeared colors or frame drops, affect video machine learning models, finding that these corruptions degrade performance on various tasks, with effects varying by task and dataset, and that standard data augmentation methods fail to recover performance.
We study how networking corruptions--data corruptions caused by networking errors--affect video machine learning (ML) models. We discover apparent networking corruptions in Kinetics-400, a benchmark video ML dataset. In a simulation study, we investigate (1) what artifacts networking corruptions cause, (2) how such artifacts affect ML models, and (3) whether standard robustness methods can mitigate their negative effects. We find that networking corruptions cause visual and temporal artifacts (i.e., smeared colors or frame drops). These networking corruptions degrade performance on a variety of video ML tasks, but effects vary by task and dataset, depending on how much temporal context the tasks require. Lastly, we evaluate data augmentation--a standard defense for data corruptions--but find that it does not recover performance.