Learning to encode motion using spatio-temporal synchrony
This addresses motion extraction in computer vision, offering a more efficient learning approach, though it appears incremental as it builds on existing motion energy models.
The paper tackles the problem of learning to extract motion from videos by detecting spatio-temporal synchrony, achieving competitive performance in motion estimation tasks with significantly reduced learning time and outperforming hand-crafted features.
We consider the task of learning to extract motion from videos. To this end, we show that the detection of spatial transformations can be viewed as the detection of synchrony between the image sequence and a sequence of features undergoing the motion we wish to detect. We show that learning about synchrony is possible using very fast, local learning rules, by introducing multiplicative "gating" interactions between hidden units across frames. This makes it possible to achieve competitive performance in a wide variety of motion estimation tasks, using a small fraction of the time required to learn features, and to outperform hand-crafted spatio-temporal features by a large margin. We also show how learning about synchrony can be viewed as performing greedy parameter estimation in the well-known motion energy model.