On Invariance and Selectivity in Representation Learning
This work addresses the challenge of designing effective representation learning methods for sensory processing, but it appears incremental as it builds on existing i-theory.
The paper tackles the problem of learning data representations that are both invariant to transformations and selective, ensuring distinct representations for non-transformed data points, and sharpens key claims of i-theory related to feedforward processing in sensory cortex.
We discuss data representation which can be learned automatically from data, are invariant to transformations, and at the same time selective, in the sense that two points have the same representation only if they are one the transformation of the other. The mathematical results here sharpen some of the key claims of i-theory -- a recent theory of feedforward processing in sensory cortex.