Hongyi Zhou

Erhan Bayraktar, Ibrahim Ekren, Hongyi Zhou

We study the uniform-in-time weak propagation of chaos for the consensus-based optimization (CBO) method on a bounded searching domain. We apply the methodology for studying long-time behaviors of interacting particle systems developed in the work of Delarue and Tse (ArXiv:2104.14973). Our work shows that the weak error has order $O(N^{-1})$ uniformly in time, where $N$ denotes the number of particles. The main strategy behind the proofs are the decomposition of the weak errors using the linearized Fokker-Planck equations and the exponential decay of their Sobolev norms. Consequently, our result leads to the joint convergence of the empirical distribution of the CBO particle system to the Dirac-delta distribution at the global minimizer in population size and running time in Wasserstein-type metrics.

Hongyi Zhou, Josiah P. Hanna, Jin Zhu et al.

This paper studies off-policy evaluation (OPE) in reinforcement learning with a focus on behavior policy estimation for importance sampling. Prior work has shown empirically that estimating a history-dependent behavior policy can lead to lower mean squared error (MSE) even when the true behavior policy is Markovian. However, the question of why the use of history should lower MSE remains open. In this paper, we theoretically demystify this paradox by deriving a bias-variance decomposition of the MSE of ordinary importance sampling (IS) estimators, demonstrating that history-dependent behavior policy estimation decreases their asymptotic variances while increasing their finite-sample biases. Additionally, as the estimated behavior policy conditions on a longer history, we show a consistent decrease in variance. We extend these findings to a range of other OPE estimators, including the sequential IS estimator, the doubly robust estimator and the marginalized IS estimator, with the behavior policy estimated either parametrically or non-parametrically.

You-Qi Nie, Jian-Yu Guan, Hongyi Zhou et al.

The randomness from a quantum random number generator (QRNG) relies on the accurate characterization of its devices. However, device imperfections and inaccurate characterizations can result in wrong entropy estimation and bias in practice, which highly affects the genuine randomness generation and may even induce the disappearance of quantum randomness in an extreme case. Here we experimentally demonstrate a measurement-device-independent (MDI) QRNG based on time-bin encoding to achieve certified quantum randomness even when the measurement devices are uncharacterized and untrusted. The MDI-QRNG is randomly switched between the regular randomness generation mode and a test mode, in which four quantum states are randomly prepared to perform measurement tomography in real-time. With a clock rate of 25 MHz, the MDI-QRNG generates a final random bit rate of 5.7 Kbps. Such implementation with an all-fiber setup provides an approach to construct a fully-integrated MDI-QRNG with trusted but error-prone devices in practice.