C. Shi

J. M. Taramón, J. P. Morgan, C. Shi et al.

We present 2-D, 3-D, and spherical mesh generators for the Finite Element Method (FEM) using triangular and tetrahedral elements. The mesh nodes are treated as if they were linked by virtual springs that obey Hooke's law. Given the desired length for the springs, the FEM is used to solve for the optimal nodal positions for the static equilibrium of this spring system. A 'guide-mesh' approach allows the user to create embedded high resolution sub-regions within a coarser mesh. The method converges rapidly. For example, in 3-D, the algorithm is able to refine a specific region within an unstructured tetrahedral spherical shell so that the edge-length factor $l_{0r}/l_{0c} = 1/33$ within a few iterations, where $l_{0r}$ and $l_{0c}$ are the desired spring length for elements inside the refined and coarse regions respectively. One use for this type of mesh is to model regional problems as a fine region within a global mesh that has no fictitious boundaries, at only a small additional computational cost. The algorithm also includes routines to locally improve the quality of the mesh and to avoid badly shaped 'slivers-like' tetrahedra.

C. Shi, S. Zhang, W. Lu et al.

Reinforcement learning is a general technique that allows an agent to learn an optimal policy and interact with an environment in sequential decision making problems. The goodness of a policy is measured by its value function starting from some initial state. The focus of this paper is to construct confidence intervals (CIs) for a policy's value in infinite horizon settings where the number of decision points diverges to infinity. We propose to model the action-value state function (Q-function) associated with a policy based on series/sieve method to derive its confidence interval. When the target policy depends on the observed data as well, we propose a SequentiAl Value Evaluation (SAVE) method to recursively update the estimated policy and its value estimator. As long as either the number of trajectories or the number of decision points diverges to infinity, we show that the proposed CI achieves nominal coverage even in cases where the optimal policy is not unique. Simulation studies are conducted to back up our theoretical findings. We apply the proposed method to a dataset from mobile health studies and find that reinforcement learning algorithms could help improve patient's health status. A Python implementation of the proposed procedure is available at https://github.com/shengzhang37/SAVE.