Zhao Liu

Wen-ming Zhang, Zhi-wei Wang, Dan-dian Feng et al.

The frequency-based method is the most commonly used method for measuring cable tension. However, the calculation formulas for the conventional frequency-based method are generally based on the ideally hinged or fixed boundary conditions without a comprehensive consideration of the inclination angle, sag-extensibility, and flexural stiffness of cables, leading to a significant error in cable tension identification. This study aimed to propose a frequency-based method of cable tension identification considering the complex boundary conditions at the two ends of cables using the particle swarm optimization (PSO) algorithm. First, the refined stay cable model was established considering the inclination angle, flexural stiffness, and sag-extensibility, as well as the rotational constraint stiffness and lateral support stiffness for the unknown boundaries of cables. The vibration mode equation of the stay cable model was discretized and solved using the finite difference method. Then, a multiparameter identification method based on the PSO algorithm was proposed. This method was able to identify the tension, flexural stiffness, axial stiffness, boundary rotational constraint stiffness, and boundary lateral support stiffness according to the measured multiorder frequencies in a synchronous manner. The feasibility and accuracy of this method were validated through numerical cases. Finally, the proposed approach was applied to the tension identification of the anchor span strands of a suspension bridge (Jindong Bridge) in China. The results of cable tension identification using the proposed method and the existing methods discussed in previous studies were compared with the on-site pressure ring measurement results. The comparison showed that the proposed approach had a high accuracy in cable tension identification.

Hua Wei, Deheng Ye, Zhao Liu et al.

Offline reinforcement learning (RL) tries to learn the near-optimal policy with recorded offline experience without online exploration. Current offline RL research includes: 1) generative modeling, i.e., approximating a policy using fixed data; and 2) learning the state-action value function. While most research focuses on the state-action function part through reducing the bootstrapping error in value function approximation induced by the distribution shift of training data, the effects of error propagation in generative modeling have been neglected. In this paper, we analyze the error in generative modeling. We propose AQL (action-conditioned Q-learning), a residual generative model to reduce policy approximation error for offline RL. We show that our method can learn more accurate policy approximations in different benchmark datasets. In addition, we show that the proposed offline RL method can learn more competitive AI agents in complex control tasks under the multiplayer online battle arena (MOBA) game Honor of Kings.

Deheng Ye, Zhao Liu, Mingfei Sun et al.

We study the reinforcement learning problem of complex action control in the Multi-player Online Battle Arena (MOBA) 1v1 games. This problem involves far more complicated state and action spaces than those of traditional 1v1 games, such as Go and Atari series, which makes it very difficult to search any policies with human-level performance. In this paper, we present a deep reinforcement learning framework to tackle this problem from the perspectives of both system and algorithm. Our system is of low coupling and high scalability, which enables efficient explorations at large scale. Our algorithm includes several novel strategies, including control dependency decoupling, action mask, target attention, and dual-clip PPO, with which our proposed actor-critic network can be effectively trained in our system. Tested on the MOBA game Honor of Kings, our AI agent, called Tencent Solo, can defeat top professional human players in full 1v1 games.

Zhao Liu, Niall Murray, Brian Lee et al.

Multiview video supports observing a scene from different viewpoints. The Joint Video Team (JVT) developed H.264/MVC to enhance the compression efficiency for multiview video, however, MVC encoded multiview video (MVC video) still requires high bitrates for transmission. This paper investigates live MVC video streaming over Peer-to-Peer (P2P) networks. The goal is to minimize the server bandwidth costs whist ensuring high streaming quality to peers. MVC employs intra-view and inter-view prediction structures, which leads to a complicated layer dependency relationship. As the peers' outbound bandwidth is shared while supplying all the MVC video layers, the bandwidth allocation to one MVC layer affects the available outbound bandwidth of the other layers. To optimise the utilisation of the peers' outbound bandwidth for providing video layers, a maximum flow based model is proposed which considers the MVC video layer dependency and the layer supplying relationship between peers. Based on the model, a layer dependency aware live MVC video streaming method over a BitTorrent-like P2P network is proposed, named MVP2P. The key components of MVP2P include a chunk scheduling strategy and a peer selection strategy for receiving peers, and a bandwidth scheduling algorithm for supplying peers. To evaluate the efficiency of the proposed solution, MVP2P is compared with existing methods considering the constraints of peer bandwidth, peer numbers, view switching rates, and peer churns. The test results show that MVP2P significantly outperforms the existing methods.