Size Matters
This work identifies a critical issue for practitioners using fully convolutional networks, revealing that input size significantly impacts performance and is not simply a 'bigger is better' scenario.
This paper investigates the performance of fully convolutional neural networks when presented with inputs of varying sizes. It finds that these networks are not size-agnostic, with each network exhibiting a preferred input size for optimal performance, and that presenting the same image at different scales can lead to different classification outcomes.
Fully convolutional neural networks can process input of arbitrary size by applying a combination of downsampling and pooling. However, we find that fully convolutional image classifiers are not agnostic to the input size but rather show significant differences in performance: presenting the same image at different scales can result in different outcomes. A closer look reveals that there is no simple relationship between input size and model performance (no `bigger is better'), but that each each network has a preferred input size, for which it shows best results. We investigate this phenomenon by applying different methods, including spectral analysis of layer activations and probe classifiers, showing that there are characteristic features depending on the network architecture. From this we find that the size of discriminatory features is critically influencing how the inference process is distributed among the layers.