Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication
This reveals a potential side-channel issue for applications like diffusion-based trajectory generation where spatial locality is critical.
The paper demonstrates that normalization layers in convolutional networks enable spatial communication far beyond local receptive fields, using a toy localization task to show iterative message passing across spatial dimensions.
This work shows that normalization layers can facilitate a surprising degree of communication across the spatial dimensions of an input tensor. We study a toy localization task with a convolutional architecture and show that normalization layers enable an iterative message passing procedure, allowing information aggregation from well outside the local receptive field. Our results suggest that normalization layers should be employed with caution in applications such as diffusion-based trajectory generation, where maintaining a spatially limited receptive field is crucial.