PREF: Predictability Regularized Neural Motion Fields
This addresses the problem of ambiguous motion estimation in dynamic scenes for vision applications, representing an incremental improvement over existing neural motion field methods.
The paper tackles the challenge of estimating 3D motion for all points in dynamic scenes from multiview data by introducing a predictability regularization method, achieving results on par or better than state-of-the-art neural motion field-based methods without prior scene knowledge.
Knowing the 3D motions in a dynamic scene is essential to many vision applications. Recent progress is mainly focused on estimating the activity of some specific elements like humans. In this paper, we leverage a neural motion field for estimating the motion of all points in a multiview setting. Modeling the motion from a dynamic scene with multiview data is challenging due to the ambiguities in points of similar color and points with time-varying color. We propose to regularize the estimated motion to be predictable. If the motion from previous frames is known, then the motion in the near future should be predictable. Therefore, we introduce a predictability regularization by first conditioning the estimated motion on latent embeddings, then by adopting a predictor network to enforce predictability on the embeddings. The proposed framework PREF (Predictability REgularized Fields) achieves on par or better results than state-of-the-art neural motion field-based dynamic scene representation methods, while requiring no prior knowledge of the scene.