Tommaso Cesari

Yue Zhang, Zhiyi Dong, Tommaso Cesari et al.

We develop a learning-theoretic framework for understanding Chain of Thought (CoT). We model CoT as the interaction between an answer map and a chain rule that generates intermediate questions autoregressively, and define the reasoning risk of a hypothesis under this interaction. Our first result is a tight canonical decomposition of this risk into two terms with opposing roles: an oracle-trajectory risk (OTR), which captures the benefit of CoT and reduces to a target-domain risk in a domain adaptation problem, and a trajectory-mismatch risk (TMR), which captures the cost of CoT through error accumulation along mismatched reasoning trajectories. We then show that this cost is unavoidable without structure: if any one of the loss, the hypothesis answer map, or the chain rule lacks stability, the TMR can be arbitrarily large even when the OTR is zero and the hypothesis is uniformly close to the ground truth. Conversely, under stability, we prove a tight upper bound on the TMR governed by an exact amplification factor that identifies bounded, linear, and exponential error-growth regimes. Together, these results give a precise theory of when CoT helps, when it hurts, and what controls the transition between the two.

Tommaso Cesari, Roberto Colomboni

We prove an effective-resistance bound for fixed-rank external-field measures. Let $d\ge2$ be an integer, let $m\in\{1,\ldots,d-1\}$. Let $w\in(0,+\infty)^d$, and let $\mathsf S$ be an $m$-element random subset of $[d]$ distributed according to the rank-$m$ external-field measure with weights $w$, i.e., \[\mathbb P(\mathsf S=S)=\frac{\prod_{i\in S}w_i}{e_m(w)},\qquad S\subseteq\{1,\dots,d\},\quad|S|=m,\] where \[e_m(w):=\sum_{\substack{T\subseteq\{1,\dots,d\}\\|T|=m}}\prod_{\ell\in T}w_\ell\] is the $m$th elementary symmetric polynomial in $w_1,\ldots,w_d$. Let $X:=(X_1,\dots,X_d)^\top$ be its indicator vector, i.e., \[X_i=\mathbb I\{i\in\mathsf S\},\qquad i\in\{1,\dots,d\}.\] Let $Σ:=\operatorname{Cov}(X)$, put $v_i:=Σ_{ii}$ for each $i\in\{1,\dots,d\}$, and let $\mathbf e_1,\ldots,\mathbf e_d$ denote the standard basis of $\mathbb R^d$. Our main result is that, for every $i\ne j$, \[(\mathbf e_i-\mathbf e_j)^\topΣ^\dagger(\mathbf e_i-\mathbf e_j)\le\frac1{v_i}+\frac1{v_j},\] where $Σ^\dagger$ is the Moore-Penrose pseudoinverse of $Σ$. As a consequence, if \[v:=(v_1,\ldots,v_d)^\top,\qquad D:=\operatorname{diag}(v),\qquad V:=\sum_{i=1}^dv_i,\] then, as a corollary, we obtain \[Σ\succeq\frac12\left(D-\frac{vv^\top}{V}\right),\] which establishes a factor-two relaxation of the normalized covariance bound conjectured by Anari, Haqi, and Ma. As a further corollary, combining our theorem with the recent framework of Anari, Haqi, and Ma yields a constant-stretch guarantee for correlated sampling on the hypersimplex without relying on the still-open normalized covariance conjecture assumed in their conditional result. Our result improves the logarithmic-in-$k$ stretch bound of Naor, Raju, Shetty, Srinivasan, Valieva, and Wajc to a constant and resolves the open question posed in their work.

Tommaso Cesari, Roberto Colomboni

We study the two-action apple-tasting problem with switching costs against an oblivious adversary. In an equivalent normalized formulation, at each round the learner chooses between a revealing action and a blind action: the revealing action gives reward $0$ and reveals the hidden value $x_t\in[-1,1]$ of the blind action; the blind action gives reward $x_t$ but reveals nothing. The learner pays one unit whenever they switches actions, and regret is measured against the best fixed action in hindsight. General feedback-graph algorithms with switching costs give $\widetilde O(T^{2/3})$ regret guarantees for this problem. The two-action apple-tasting graph was the natural candidate for the missing $Ω(T^{2/3})$ obstruction in the switching-cost classification: such a lower bound would have transferred to a large family of still-unclassified feedback graphs. We prove that this obstruction is not there: the oblivious minimax expected regret for this problem satisfies \[ \frac{1}{2\sqrt3}\cdot\sqrt T \le R_T^\star \le 2\sqrt{3}\cdot \sqrt{T}. \]

Tommaso Cesari, Roberto Colomboni

We study stochastic decision-theoretic online learning with full information and event-level pure differential privacy. A COLT open problem of Hu and Mehta asks to determine the optimal gap-dependent regret rate for stochastic decision-theoretic online learning under pure event-level differential privacy. For $K$ actions, losses in $[0,1]$, and a unique best action separated from the second-best action by gap $Δ_{\min}$, the known lower bound is of order $ \frac{\log K}{\min\{Δ_{\min},\varepsilon\}}, $ or equivalently, up to universal constants, of order \[ \frac{\log K}{Δ_{\min}}+\frac{\log K}{\varepsilon}. \] We give a horizon-free pure-DP algorithm and prove the explicit regret bound \[ \operatorname{Reg}_T \le 1000 \cdot \left(\frac{\log K}{Δ_{\min}}+\frac{\log K}{\varepsilon}\right) \] for every horizon $T$. The numerical constant is not optimized. The algorithm partitions time into blocks of exponentially increasing size, plays a single action throughout each block, and chooses the next action by an exponential mechanism applied to a data-independent random prefix of the previous block. The random prefix converts block regret into a sum, over all prefix lengths, of softmax selection errors. A single entropy-potential argument controls all privacy-dominated large-gap actions at cost $\log K/\varepsilon$.

Guglielmo Beretta, Tommaso Cesari, Roberto Colomboni et al.

We study the last iterate of the stochastic subgradient method for one-dimensional convex Lipschitz objectives. For a fixed horizon $n$, we consider the standard fixed stepsizes $η=Θ(1/\sqrt n)$. We prove that, for such stepsize policies, under additive i.i.d. subgradient noise with uniformly bounded variance, the last iterate features an optimization error of order $1/\sqrt n$, thereby removing the extra $(\log n)$ factor present in existing generic bounds. On the other hand, we show that without the i.i.d. assumption, the optimization error can be of order $(\log n)/\sqrt n$. Thus, under the uniformly bounded variance assumption alone, the last iterate of SsGM is suboptimal even in dimension one, resolving negatively an open problem posed in Koren and Segal, COLT, 2020.