Shift-Memory Network for Temporal Scene Segmentation
This work addresses the need for efficient semantic segmentation on edge devices, though it appears incremental as it builds on existing encoding-decoding baselines.
The paper tackles the problem of redundant computation in temporal scene segmentation for real-time applications by proposing a Shift-Memory Network (SMN) that reuses network values without accuracy loss, achieving equivalent accuracy to shift-mode methods but with faster inference speeds and smaller memory usage.
Semantic segmentation has achieved great accuracy in understanding spatial layout. For real-time tasks based on dynamic scenes, we extend semantic segmentation in temporal domain to enhance the spatial accuracy with motion. We utilize a shift-mode network over streaming input to ensure zero-latency output. For the data overlap under shifting network, this paper identifies repeated computation in fixed periods across network layers. To avoid this redundancy, we derive a Shift-Memory Network (SMN) from encoding-decoding baseline to reuse the network values without accuracy loss. Trained in patch-mode, the SMN extracts the network parameters for SMN to perform inference promptly in compact memory. We segment dynamic scenes from 1D scanning input and 2D video. The experiments of SMN achieve equivalent accuracy as shift-mode but in faster inference speeds and much smaller memory. This will facilitate semantic segmentation in real-time application on edge devices.