Texture Interpolation for Probing Visual Perception
This work addresses the gap in applying deep texture synthesis to study visual perception, offering a method to probe human and animal visual systems, though it is incremental as it builds on existing CNN-based approaches.
The authors tackled the problem of understanding why deep texture synthesis works well and applied it to probe visual perception, showing that CNN activations follow elliptical distributions and using optimal transport to interpolate textures, which matched perceptual geometry better than other methods and was validated through human and macaque experiments.
Texture synthesis models are important tools for understanding visual processing. In particular, statistical approaches based on neurally relevant features have been instrumental in understanding aspects of visual perception and of neural coding. New deep learning-based approaches further improve the quality of synthetic textures. Yet, it is still unclear why deep texture synthesis performs so well, and applications of this new framework to probe visual perception are scarce. Here, we show that distributions of deep convolutional neural network (CNN) activations of a texture are well described by elliptical distributions and therefore, following optimal transport theory, constraining their mean and covariance is sufficient to generate new texture samples. Then, we propose the natural geodesics (ie the shortest path between two points) arising with the optimal transport metric to interpolate between arbitrary textures. Compared to other CNN-based approaches, our interpolation method appears to match more closely the geometry of texture perception, and our mathematical framework is better suited to study its statistical nature. We apply our method by measuring the perceptual scale associated to the interpolation parameter in human observers, and the neural sensitivity of different areas of visual cortex in macaque monkeys.