A chain rule for the expected suprema of Gaussian processes
This work addresses a theoretical bottleneck in analyzing learning algorithms with composite hypotheses, but it is incremental as it builds on existing Gaussian process theory.
The paper tackles the problem of bounding the expected supremum of Gaussian processes indexed by composite classes, such as those in multi-layer models, by deriving a chain rule that separates properties of the index set and function class.
The expected supremum of a Gaussian process indexed by the image of an index set under a function class is bounded in terms of separate properties of the index set and the function class. The bound is relevant to the estimation of nonlinear transformations or the analysis of learning algorithms whenever hypotheses are chosen from composite classes, as is the case for multi-layer models.