Wentuo Fang

Haohui Chen, Zhiyong Chen, Aoxiang Liu et al.

To obtain better value estimation in reinforcement learning, we propose a novel algorithm based on the double actor-critic framework with temporal difference error-driven regularization, abbreviated as TDDR. TDDR employs double actors, with each actor paired with a critic, thereby fully leveraging the advantages of double critics. Additionally, TDDR introduces an innovative critic regularization architecture. Compared to classical deterministic policy gradient-based algorithms that lack a double actor-critic structure, TDDR provides superior estimation. Moreover, unlike existing algorithms with double actor-critic frameworks, TDDR does not introduce any additional hyperparameters, significantly simplifying the design and implementation process. Experiments demonstrate that TDDR exhibits strong competitiveness compared to benchmark algorithms in challenging continuous control tasks.

Haohui Chen, Zhiyong Chen, Aoxiang Liu et al.

Deterministic policy gradient algorithms for continuous control suffer from value estimation biases that degrade performance. While double critics reduce such biases, the exploration potential of double actors remains underexplored. Building on temporal-difference error-driven regularization (TDDR), a double actor-critic framework, this work introduces enhanced methods to achieve flexible bias control and stronger representation learning. We propose three convex combination strategies, symmetric and asymmetric, that balance pessimistic estimates to mitigate overestimation and optimistic exploration via double actors to alleviate underestimation. A single hyperparameter governs this mechanism, enabling tunable control across the bias spectrum. To further improve performance, we integrate augmented state and action representations into the actor and critic networks. Extensive experiments show that our approach consistently outperforms benchmarks, demonstrating the value of tunable bias and revealing that both overestimation and underestimation can be exploited differently depending on the environment.

Wentuo Fang, Mohsen Zamani, Zhiyong Chen

This paper aims at secure and privacy preserving consensus algorithms of networked systems. Due to the technical challenges behind decentralized design of such algorithms, the existing results are mainly restricted to a network of systems with simplest first-order dynamics. Like many other control problems, breakthrough of the gap between first-order dynamics and higher-order ones demands for more advanced technical developments. In this paper, we explore a Paillier encryption based average consensus algorithm for a network of systems with second-order dynamics, with randomness added to network weights. The conditions for privacy preserving, especially depending on consensus rate, are thoroughly studied with theoretical analysis and numerical verification.