A resource-efficient model for deep kernel learning
This addresses computational efficiency for machine learning practitioners, but appears incremental as it builds on existing decomposition methods.
The paper tackles the curse of dimensionality in deep kernel learning by proposing a model-level decomposition approach that combines operator and network decomposition, with results analyzed for accuracy and scalability.
According to the Hughes phenomenon, the major challenges encountered in computations with learning models comes from the scale of complexity, e.g. the so-called curse of dimensionality. There are various approaches for accelerate learning computations with minimal loss of accuracy. These approaches range from model-level to implementation-level approaches. To the best of our knowledge, the first one is rarely used in its basic form. Perhaps, this is due to theoretical understanding of mathematical insights of model decomposition approaches, and thus the ability of developing mathematical improvements has lagged behind. We describe a model-level decomposition approach that combines both the decomposition of the operators and the decomposition of the network. We perform a feasibility analysis on the resulting algorithm, both in terms of its accuracy and scalability.