Recurrent Flow-Guided Semantic Forecasting
This addresses real-time prediction needs for autonomous systems, such as forecasting pedestrian trajectories, with incremental improvements in efficiency.
The paper tackles semantic forecasting for autonomous vehicles by decomposing it into segmentation and optical flow prediction, achieving state-of-the-art accuracy on short-term and moving objects while reducing model parameters by up to 95% and increasing efficiency by over 40x.
Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key to their success. Semantic anticipation is a relatively under-explored area for which autonomous vehicles could take advantage of (e.g., forecasting pedestrian trajectories). Motivated by the need for real-time prediction in autonomous systems, we propose to decompose the challenging semantic forecasting task into two subtasks: current frame segmentation and future optical flow prediction. Through this decomposition, we built an efficient, effective, low overhead model with three main components: flow prediction network, feature-flow aggregation LSTM, and end-to-end learnable warp layer. Our proposed method achieves state-of-the-art accuracy on short-term and moving objects semantic forecasting while simultaneously reducing model parameters by up to 95% and increasing efficiency by greater than 40x.