Bounded-Rationality, Hedging, and Generalization
For machine learning theorists, it provides a new information-geometric framework linking bounded rationality, hedging, and generalization, though the results are theoretical and not yet empirically validated.
The paper models learning as a bounded-rational decision problem, showing that a learner's response law induces tradeoff curves between training loss and sample dependence. It demonstrates that generalization can be treated as a testable hedging property, recoverable from black-box behavior via loss perturbations.
A learner does not only fit data; it also determines how strongly the training sample may shape its output and how much distortion it can hedge. We study this relation as a bounded-rational decision problem whose primitive object is the induced channel from samples to outputs. The learner's response law determines which changes in this channel are cheap or costly, and therefore induces both a lower tradeoff curve between training loss and sample dependence and a matched upper certificate curve. When the response law is represented by an $f$-divergence regularizer, these curves live in the regularizer's native information geometry, with KL as the special case corresponding to Shannon mutual information. We show how the hedge and the two curves can be recovered from black-box behavior by observing responses to scaled losses and local loss perturbations. In learning, population loss is empirical loss plus the distortion induced by the particular training sample. The recovered hedge gives a practical certificate when it covers that distortion. Thus generalization is treated as a testable hedging property of the learner's own response law.