Adaptive Sampling for Convex Regression
This addresses a problem in behavioral and social sciences where convex function learning is common, offering a novel adaptive approach that is not incremental.
The paper tackles the problem of learning convex functions in the L∞ norm by introducing the first principled adaptive-sampling procedure, which nearly attains the information-theoretically optimal error rate for each function and substantially outperforms uniform sampling in low-noise settings with large budgets.
In this paper, we introduce the first principled adaptive-sampling procedure for learning a convex function in the $L_\infty$ norm, a problem that arises often in the behavioral and social sciences. We present a function-specific measure of complexity and use it to prove that, for each convex function $f_{\star}$, our algorithm nearly attains the information-theoretically optimal, function-specific error rate. We also corroborate our theoretical contributions with numerical experiments, finding that our method substantially outperforms passive, uniform sampling for favorable synthetic and data-derived functions in low-noise settings with large sampling budgets. Our results also suggest an idealized "oracle strategy", which we use to gauge the potential advance of any adaptive-sampling strategy over passive sampling, for any given convex function.