Yue Yu

Shenghui Chen, Yue Yu, David Fridovich-Keil et al.

Markov games model interactions among multiple players in a stochastic, dynamic environment. Each player in a Markov game maximizes its expected total discounted reward, which depends upon the policies of the other players. We formulate a class of Markov games, termed affine Markov games, where an affine reward function couples the players' actions. We introduce a novel solution concept, the soft-Bellman equilibrium, where each player is boundedly rational and chooses a soft-Bellman policy rather than a purely rational policy as in the well-known Nash equilibrium concept. We provide conditions for the existence and uniqueness of the soft-Bellman equilibrium and propose a nonlinear least-squares algorithm to compute such an equilibrium in the forward problem. We then solve the inverse game problem of inferring the players' reward parameters from observed state-action trajectories via a projected-gradient algorithm. Experiments in a predator-prey OpenAI Gym environment show that the reward parameters inferred by the proposed algorithm outperform those inferred by a baseline algorithm: they reduce the Kullback-Leibler divergence between the equilibrium policies and observed policies by at least two orders of magnitude.

Xunpeng Huang, Xianfeng Liang, Zhengyang Liu et al.

Second-order optimization methods have desirable convergence properties. However, the exact Newton method requires expensive computation for the Hessian and its inverse. In this paper, we propose SPAN, a novel approximate and fast Newton method. SPAN computes the inverse of the Hessian matrix via low-rank approximation and stochastic Hessian-vector products. Our experiments on multiple benchmark datasets demonstrate that SPAN outperforms existing first-order and second-order optimization methods in terms of the convergence wall-clock time. Furthermore, we provide a theoretical analysis of the per-iteration complexity, the approximation error, and the convergence rate. Both the theoretical analysis and experimental results show that our proposed method achieves a better trade-off between the convergence rate and the per-iteration efficiency.

Hongliang Liang, Mingyu Li, Qiong Zhang et al.

Intel SGX provisions shielded executions for security-sensitive computation, but lacks support for trusted system services (TSS), such as clock, network and filesystem. This makes \textit{enclaves} vulnerable to Iago attacks~\cite{DBLP:conf/asplos/CheckowayS13} in the face of a powerful malicious system. To mitigate this problem, we present Aurora, a novel architecture that provides TSSes via a secure channel between enclaves and devices on top of an untrusted system, and implement two types of TSSes, i.e. clock and end-to-end network. We evaluate our solution by porting SQLite and OpenSSL into Aurora, experimental results show that SQLite benefits from a \textit{microsecond} accuracy trusted clock and OpenSSL gains end-to-end secure network with about 1ms overhead.