Average-case Complexity of Teaching Convex Polytopes via Halfspace Queries
This work addresses fundamental machine learning problems like perceptron training and learning separable dichotomies, offering insights into teaching and learning complexities, though it appears incremental by generalizing known bounds.
The paper tackles the problem of teaching a learner to locate a target region defined by intersections of halfspaces, showing that the average-case teaching complexity is Θ(d), contrasting with the worst-case complexity of Θ(n). It also provides bounds for average-case learning complexity as Θ(n) for i.i.d. queries and Θ(d log(n)) for active queries.
We examine the task of locating a target region among those induced by intersections of $n$ halfspaces in $\mathbb{R}^d$. This generic task connects to fundamental machine learning problems, such as training a perceptron and learning a $φ$-separable dichotomy. We investigate the average teaching complexity of the task, i.e., the minimal number of samples (halfspace queries) required by a teacher to help a version-space learner in locating a randomly selected target. As our main result, we show that the average-case teaching complexity is $Θ(d)$, which is in sharp contrast to the worst-case teaching complexity of $Θ(n)$. If instead, we consider the average-case learning complexity, the bounds have a dependency on $n$ as $Θ(n)$ for \tt{i.i.d.} queries and $Θ(d \log(n))$ for actively chosen queries by the learner. Our proof techniques are based on novel insights from computational geometry, which allow us to count the number of convex polytopes and faces in a Euclidean space depending on the arrangement of halfspaces. Our insights allow us to establish a tight bound on the average-case complexity for $φ$-separable dichotomies, which generalizes the known $\mathcal{O}(d)$ bound on the average number of "extreme patterns" in the classical computational geometry literature (Cover, 1965).