Neural TMDlayer: Modeling Instantaneous flow of features via SDE Generators
This work addresses computer vision tasks by offering an incremental improvement through a simple, efficient plug-in layer derived from SDE theory.
The authors tackled the problem of improving computer vision algorithms by introducing a plug-in layer based on stochastic differential equations (SDEs) to model instantaneous feature flow, resulting in efficiency or performance gains in tasks like few-shot learning, point cloud transformers, and deep variational segmentation.
We study how stochastic differential equation (SDE) based ideas can inspire new modifications to existing algorithms for a set of problems in computer vision. Loosely speaking, our formulation is related to both explicit and implicit strategies for data augmentation and group equivariance, but is derived from new results in the SDE literature on estimating infinitesimal generators of a class of stochastic processes. If and when there is nominal agreement between the needs of an application/task and the inherent properties and behavior of the types of processes that we can efficiently handle, we obtain a very simple and efficient plug-in layer that can be incorporated within any existing network architecture, with minimal modification and only a few additional parameters. We show promising experiments on a number of vision tasks including few shot learning, point cloud transformers and deep variational segmentation obtaining efficiency or performance improvements.