Xinkai Li

Chris Beeler, Sriram Ganapathi Subramanian, Kyle Sprague et al.

This paper provides a simulated laboratory for making use of Reinforcement Learning (RL) for chemical discovery. Since RL is fairly data intensive, training agents `on-the-fly' by taking actions in the real world is infeasible and possibly dangerous. Moreover, chemical processing and discovery involves challenges which are not commonly found in RL benchmarks and therefore offer a rich space to work in. We introduce a set of highly customizable and open-source RL environments, ChemGymRL, based on the standard Open AI Gym template. ChemGymRL supports a series of interconnected virtual chemical benches where RL agents can operate and train. The paper introduces and details each of these benches using well-known chemical reactions as illustrative examples, and trains a set of standard RL algorithms in each of these benches. Finally, discussion and comparison of the performances of several standard RL methods are provided in addition to a list of directions for future work as a vision for the further development and usage of ChemGymRL.

Chris Beeler, Xinkai Li, Colin Bellinger et al.

Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Markov decision processes (MDPs). Adding in controlled sensing methods, we show that these policies can also lower measurement costs at the same time.

Jianming Liu, Xinkai Li

In this paper, a novel Hermite radial basis function-based differential quadrature method (H-RBF-DQ) is presented. This new method is designed to treat derivative boundary conditions accurately. The developed method is very different from the original Hermite RBF method. In order to illustrate the specific process of this method, although the method can be used to study most of partial differential equations, the numerical simulation of two-dimensional variable-order time fractional advection-diffusion equation is chosen as an example. For the general case of irregular geometry, the meshless local form of RBF-DQ was used and the multiquadric type of radial basis functions are selected for the computations. The method is validated by the documented test examples involving variable-order fractional modeling of air pollution. The numerical results demonstrate the robustness and the versatility of the proposed approach.