The Teaching Dimension of Kernel Perceptron
This addresses the problem of algorithmic machine teaching for nonlinear learners, which has been largely unexplored compared to linear settings.
The paper establishes the teaching dimension (sample complexity) for kernelized perceptrons, showing it is Θ(d) for linear perceptrons and Θ(d^k) for polynomial kernel perceptrons, and provides a bound for Gaussian kernel perceptrons under smooth data assumptions.
Algorithmic machine teaching has been studied under the linear setting where exact teaching is possible. However, little is known for teaching nonlinear learners. Here, we establish the sample complexity of teaching, aka teaching dimension, for kernelized perceptrons for different families of feature maps. As a warm-up, we show that the teaching complexity is $Θ(d)$ for the exact teaching of linear perceptrons in $\mathbb{R}^d$, and $Θ(d^k)$ for kernel perceptron with a polynomial kernel of order $k$. Furthermore, under certain smooth assumptions on the data distribution, we establish a rigorous bound on the complexity for approximately teaching a Gaussian kernel perceptron. We provide numerical examples of the optimal (approximate) teaching set under several canonical settings for linear, polynomial and Gaussian kernel perceptrons.