Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks
This work addresses a foundational problem in statistical learning theory for researchers, providing theoretical bounds that could enhance transfer learning methods, though it appears incremental as it builds on existing VC class frameworks.
The paper tackles the problem of estimating a prior distribution over a VC class from independent learning tasks, deriving upper and lower bounds on the optimal convergence rates under smoothness conditions, with the number of samples per dataset set to the VC dimension. The results show implications for transfer learning improvements and extend to real-valued functions with consistency proofs and applications in algorithmic economics.
We study the optimal rates of convergence for estimating a prior distribution over a VC class from a sequence of independent data sets respectively labeled by independent target functions sampled from the prior. We specifically derive upper and lower bounds on the optimal rates under a smoothness condition on the correct prior, with the number of samples per data set equal the VC dimension. These results have implications for the improvements achievable via transfer learning. We additionally extend this setting to real-valued function, where we establish consistency of an estimator for the prior, and discuss an additional application to a preference elicitation problem in algorithmic economics.