Convolutional networks and learning invariant to homogeneous multiplicative scalings
This addresses robustness issues in supervised learning for convnets, though it appears incremental as it modifies only the classification stage.
The paper tackles the problem of improving classification robustness in convolutional networks by replacing multinomial logistic regression with a scale-invariant classification stage, resulting in slightly lower errors on standard test sets.
The conventional classification schemes -- notably multinomial logistic regression -- used in conjunction with convolutional networks (convnets) are classical in statistics, designed without consideration for the usual coupling with convnets, stochastic gradient descent, and backpropagation. In the specific application to supervised learning for convnets, a simple scale-invariant classification stage turns out to be more robust than multinomial logistic regression, appears to result in slightly lower errors on several standard test sets, has similar computational costs, and features precise control over the actual rate of learning. "Scale-invariant" means that multiplying the input values by any nonzero scalar leaves the output unchanged.