Weiya Ren

Weiya Ren

In this paper, we consider the problem of actor-critic reinforcement learning. Firstly, we extend the actor-critic architecture to actor-critic-N architecture by introducing more critics beyond rewards. Secondly, we combine the reward-based critic with a potential-field-based critic to formulate the proposed potential field guided actor-critic reinforcement learning approach (actor-critic-2). This can be seen as a combination of the model-based gradients and the model-free gradients in policy improvement. State with large potential field often contains a strong prior information, such as pointing to the target at a long distance or avoiding collision by the side of an obstacle. In this situation, we should trust potential-field-based critic more as policy evaluation to accelerate policy improvement, where action policy tends to be guided. For example, in practical application, learning to avoid obstacles should be guided rather than learned by trial and error. State with small potential filed is often lack of information, for example, at the local minimum point or around the moving target. At this time, we should trust reward-based critic as policy evaluation more to evaluate the long-term return. In this case, action policy tends to explore. In addition, potential field evaluation can be combined with planning to estimate a better state value function. In this way, reward design can focus more on the final stage of reward, rather than reward shaping or phased reward. Furthermore, potential field evaluation can make up for the lack of communication in multi-agent cooperation problem, i.e., multi-agent each has a reward-based critic and a relative unified potential-field-based critic with prior information. Thirdly, simplified experiments on predator-prey game demonstrate the effectiveness of the proposed approach.

Weiya Ren

In this paper, we consider the problem of large scale multi agent reinforcement learning. Firstly, we studied the representation problem of the pairwise value function to reduce the complexity of the interactions among agents. Secondly, we adopt a l2-norm trick to ensure the trivial term of the approximated value function is bounded. Thirdly, experimental results on battle game demonstrate the effectiveness of the proposed approach.

Weiya Ren

Collectiveness motions of crowd systems have attracted a great deal of attentions in recently years. In this paper, we try to measure the collectiveness of a crowd system by the proposed node clique learning method. The proposed method is a graph based method, and investigates the influence from one node to other nodes. A node is represented by a set of nodes which named a clique, which is obtained by spreading information from this node to other nodes in graph. Then only nodes with sufficient information are selected as the clique of this node. The motion coherence between two nodes is defined by node cliques comparing. The collectiveness of a node and the collectiveness of the crowd system are defined by the nodes coherence. Self-driven particle (SDP) model and the crowd motion database are used to test the ability of the proposed method in measuring collectiveness.

Wei-Ya Ren, Shuo-Hao Li, Qiang Guo et al.

The key in agglomerative clustering is to define the affinity measure between two sets. A novel agglomerative clustering method is proposed by utilizing the path integral to define the affinity measure. Firstly, the path integral descriptor of an edge, a node and a set is computed by path integral and exponent generating function. Then, the affinity measure between two sets is obtained by path integral descriptor of sets. Several good properties of the path integral descriptor is proposed in this paper. In addition, we give the physical interpretation of the proposed path integral descriptor of a set. The proposed path integral descriptor of a set can be regard as the collectiveness measure of a set, which can be a moving system such as human crowd, sheep herd and so on. Self-driven particle (SDP) model is used to test the ability of the proposed method in measuring collectiveness.