Functionally Modular and Interpretable Temporal Filtering for Robust Segmentation
This work addresses robustness in vision-based perception for autonomous systems, but it is incremental as it builds on existing temporal filtering methods with modular and interpretable improvements.
The paper tackled the problem of robust segmentation in autonomous systems under challenging conditions like sensor outages or heavy weather by proposing a functionally modular temporal filter that stabilizes feature representations using previous time steps, showing improved ability to overcome data-inherent perturbations in experiments with photorealistic synthetic video data.
The performance of autonomous systems heavily relies on their ability to generate a robust representation of the environment. Deep neural networks have greatly improved vision-based perception systems but still fail in challenging situations, e.g. sensor outages or heavy weather. These failures are often introduced by data-inherent perturbations, which significantly reduce the information provided to the perception system. We propose a functionally modularized temporal filter, which stabilizes an abstract feature representation of a single-frame segmentation model using information of previous time steps. Our filter module splits the filter task into multiple less complex and more interpretable subtasks. The basic structure of the filter is inspired by a Bayes estimator consisting of a prediction and an update step. To make the prediction more transparent, we implement it using a geometric projection and estimate its parameters. This additionally enables the decomposition of the filter task into static representation filtering and low-dimensional motion filtering. Our model can cope with missing frames and is trainable in an end-to-end fashion. Using photorealistic, synthetic video data, we show the ability of the proposed architecture to overcome data-inherent perturbations. The experiments especially highlight advantages introduced by an interpretable and explicit filter module.