Moise Blanchard

Yan Dai, Moise Blanchard, Patrick Jaillet

We study repeated resource allocation with strategic agents, where monetary transfers are disallowed and the planner has no prior information on agents' utility distributions. Inspired by the costly state verification literature, we assume the planner can request costly audits on the winning agent after allocation, revealing their true utility but without the ability to revoke the allocation. We design a mechanism achieving $T$-independent $\mathcal O(K^2)$ regret in social welfare while requesting $\mathcal O(K^3 \log T)$ audits in expectation, where $K$ is the number of agents and $T$ is the number of rounds. We further show an $Ω(K)$ lower bound on the regret and an $Ω(1)$ lower bound on the number of audits required for low regret. We also generalize our mechanism and analysis to imperfect audit models. Algorithmically, we show that incentivizing truthful behavior relies on accurately estimating agents' truthful winning probability online. To achieve this, we impose future punishments via adaptive audits; we also introduce an incentive-aligned flagging component allowing agents to flag biased estimates, which we prove is in their best interest. Analytically, without distributional information, the revelation principle cannot dictate a truth-telling equilibrium. Instead, we characterize a Perfect Bayesian Equilibrium via a reduction to an auxiliary game with only benign strategies. The technical tools developed herein can be of independent interest for other robust mechanism design problems where the revelation principle is inapplicable.

Moise Blanchard, Steve Hanneke, Patrick Jaillet

We study the fundamental limits of learning in contextual bandits, where a learner's rewards depend on their actions and a known context, which extends the canonical multi-armed bandit to the case where side-information is available. We are interested in universally consistent algorithms, which achieve sublinear regret compared to any measurable fixed policy, without any function class restriction. For stationary contextual bandits, when the underlying reward mechanism is time-invariant, Blanchard et. al (2022) characterized learnable context processes for which universal consistency is achievable; and further gave algorithms ensuring universal consistency whenever this is achievable, a property known as optimistic universal consistency. It is well understood, however, that reward mechanisms can evolve over time, possibly adversarially, and depending on the learner's actions. We show that optimistic universal learning for contextual bandits with adversarial rewards is impossible in general, contrary to all previously studied settings in online learning -- including standard supervised learning. We also give necessary and sufficient conditions for universal learning under various adversarial reward models, and an exact characterization for online rewards. In particular, the set of learnable processes for these reward models is still extremely general -- larger than i.i.d., stationary or ergodic -- but in general strictly smaller than that for supervised learning or stationary contextual bandits, shedding light on new adversarial phenomena.

Moise Blanchard, Steve Hanneke, Patrick Jaillet

We consider the contextual bandit problem on general action and context spaces, where the learner's rewards depend on their selected actions and an observable context. This generalizes the standard multi-armed bandit to the case where side information is available, e.g., patients' records or customers' history, which allows for personalized treatment. We focus on consistency -- vanishing regret compared to the optimal policy -- and show that for large classes of non-i.i.d. contexts, consistency can be achieved regardless of the time-invariant reward mechanism, a property known as universal consistency. Precisely, we first give necessary and sufficient conditions on the context-generating process for universal consistency to be possible. Second, we show that there always exists an algorithm that guarantees universal consistency whenever this is achievable, called an optimistically universal learning rule. Interestingly, for finite action spaces, learnable processes for universal learning are exactly the same as in the full-feedback setting of supervised learning, previously studied in the literature. In other words, learning can be performed with partial feedback without any generalization cost. The algorithms balance a trade-off between generalization (similar to structural risk minimization) and personalization (tailoring actions to specific contexts). Lastly, we consider the case of added continuity assumptions on rewards and show that these lead to universal consistency for significantly larger classes of data-generating processes.