TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video Scene Parsing
This work addresses video scene parsing for applications like autonomous driving, but it appears incremental as it builds on existing methods for a specific dataset.
The paper tackles video scene parsing in diverse real-world scenarios by proposing TBN-ViT, a Temporal Bilateral Network with Vision Transformer, achieving a mean intersection over union (mIoU) of 49.85% on the VSPW2021 Challenge test dataset.
Video scene parsing in the wild with diverse scenarios is a challenging and great significance task, especially with the rapid development of automatic driving technique. The dataset Video Scene Parsing in the Wild(VSPW) contains well-trimmed long-temporal, dense annotation and high resolution clips. Based on VSPW, we design a Temporal Bilateral Network with Vision Transformer. We first design a spatial path with convolutions to generate low level features which can preserve the spatial information. Meanwhile, a context path with vision transformer is employed to obtain sufficient context information. Furthermore, a temporal context module is designed to harness the inter-frames contextual information. Finally, the proposed method can achieve the mean intersection over union(mIoU) of 49.85\% for the VSPW2021 Challenge test dataset.