Understanding image motion with group representations
This addresses the challenge of underconstrained motion learning for agents in dynamic environments, offering a label-free approach that could benefit applications like localization and tracking, though it appears incremental in leveraging group theory for constraints.
The paper tackled the problem of learning motion representations from unlabeled video by proposing a model based on group properties of transformations, and demonstrated that a deep neural network trained with this method captures motion in synthetic 2D and real-world vehicle sequences without labels.
Motion is an important signal for agents in dynamic environments, but learning to represent motion from unlabeled video is a difficult and underconstrained problem. We propose a model of motion based on elementary group properties of transformations and use it to train a representation of image motion. While most methods of estimating motion are based on pixel-level constraints, we use these group properties to constrain the abstract representation of motion itself. We demonstrate that a deep neural network trained using this method captures motion in both synthetic 2D sequences and real-world sequences of vehicle motion, without requiring any labels. Networks trained to respect these constraints implicitly identify the image characteristic of motion in different sequence types. In the context of vehicle motion, this method extracts information useful for localization, tracking, and odometry. Our results demonstrate that this representation is useful for learning motion in the general setting where explicit labels are difficult to obtain.