Inheritance Between Feedforward and Convolutional Networks via Model Projection
This work addresses the problem of parameter efficiency in transfer learning for computer vision researchers, though it is incremental as it builds on existing network formalizations.
The paper tackles the problem of inefficient parameter reuse between feedforward and convolutional networks by introducing model projection, a transfer learning method that reduces trained parameters in convolutional layers by up to 99% while maintaining adaptability, achieving competitive performance on downstream image classification tasks.
Techniques for feedforward networks (FFNs) and convolutional networks (CNNs) are frequently reused across families, but the relationship between the underlying model classes is rarely made explicit. We introduce a unified node-level formalization with tensor-valued activations and show that generalized feedforward networks form a strict subset of generalized convolutional networks. Motivated by the mismatch in per-input parameterization between the two families, we propose model projection, a parameter-efficient transfer learning method for CNNs that freezes pretrained per-input-channel filters and learns a single scalar gate for each (output channel, input channel) contribution. Projection keeps all convolutional layers adaptable to downstream tasks while substantially reducing the number of trained parameters in convolutional layers. We prove that projected nodes take the generalized FFN form, enabling projected CNNs to inherit feedforward techniques that do not rely on homogeneous layer inputs. Experiments across multiple ImageNet-pretrained backbones and several downstream image classification datasets show that model projection is a strong transfer learning baseline under simple training recipes.