S. Ivan Trapasso

Fabio Nicola, S. Ivan Trapasso

Within the mathematical analysis of deep convolutional neural networks, the wavelet scattering transform introduced by Stéphane Mallat is a unique example of how the ideas of multiscale analysis can be combined with a cascade of modulus nonlinearities to build a nonexpansive, translation invariant signal representation with provable geometric stability properties, namely Lipschitz continuity to the action of small $C^2$ diffeomorphisms - a remarkable result for both theoretical and practical purposes, inherently depending on the choice of the filters and their arrangement into a hierarchical architecture. In this note, we further investigate the intimate relationship between the scattering structure and the regularity of the deformation in the Hölder regularity scale $C^α$, $α>0$. We are able to precisely identify the stability threshold, proving that stability is still achievable for deformations of class $C^α$, $α>1$, whereas instability phenomena can occur at lower regularity levels modelled by $C^α$, $0\le α<1$. While the behaviour at the threshold given by Lipschitz (or even $C^1$) regularity remains beyond reach, we are able to prove a stability bound in that case, up to $\varepsilon$ losses.

Fabio Nicola, S. Ivan Trapasso

Motivated by the problem of robustness to deformations of the input for deep convolutional neural networks, we identify signal classes which are inherently stable to irregular deformations induced by distortion fields $τ\in L^\infty(\mathbb{R}^d;\mathbb{R}^d)$, to be characterized in terms of a generalized modulus of continuity associated with the deformation operator. Resorting to ideas of harmonic and multiscale analysis, we prove that for signals in multiresolution approximation spaces $U_s$ at scale $s$, stability in $L^2$ holds in the regime $\|τ\|_{L^\infty}/s\ll 1$ - essentially as an effect of the uncertainty principle. Instability occurs when $\|τ\|_{L^\infty}/s\gg 1$, and we provide a sharp upper bound for the asymptotic growth rate. The stability results are then extended to signals in the Besov space $B^{d/2}_{2,1}$ tailored to the given multiresolution approximation. We also consider the case of more general time-frequency deformations. Finally, we provide stochastic versions of the aforementioned results, namely we study the issue of stability in mean when $τ(x)$ is modeled as a random field (not bounded, in general) with identically distributed variables $|τ(x)|$, $x\in\mathbb{R}^d$.