Rajdeep Dutta

Rajdeep Dutta, Daniel Pack

In this paper, we present the benefits of exploring different topologies with equal connectivity measure, or iso-connectivity topologies, in relation to the multiagent system dynamics. The level of global information sharing ability among agents in a multi-agent network can be quantified by a connectivity measure, called as the Algebraic Connectivity of the associated graph consisting of point-mass agents as nodes and inter-connection links between them as edges. Distinct topologies with the same connectivity play profound role in multi-agent dynamics as they offer various ways to reorganize agents locations according to the requirement during a cooperative mission, without sacrificing the information exchange capability of the entire network. Determination of the distinct multi-agent graphs with identical connectivity is a multimodal problem, in other words, there exist multiple graphs that share the same connectivity. We present analytical solutions of determining distinct graphs with identical connectivity. A family of isospectral graphs are found out by utilizing an appropriate similarity transformation. Moreover, a zone of no connectivity change in a dense graph is unraveled where an agent can move freely without causing any change in the global connectivity. The proposed solutions are validated with the help of sufficient examples.

J. Senthilnath, Bangjian Zhou, Zhen Wei Ng et al.

In the realm of sequential decision-making tasks, the exploration capability of a reinforcement learning (RL) agent is paramount for achieving high rewards through interactions with the environment. To enhance this crucial ability, we propose SAQN, a novel approach wherein a self-evolving autoencoder (SA) is embedded with a Q-Network (QN). In SAQN, the self-evolving autoencoder architecture adapts and evolves as the agent explores the environment. This evolution enables the autoencoder to capture a diverse range of raw observations and represent them effectively in its latent space. By leveraging the disentangled states extracted from the encoder generated latent space, the QN is trained to determine optimal actions that improve rewards. During the evolution of the autoencoder architecture, a bias-variance regulatory strategy is employed to elicit the optimal response from the RL agent. This strategy involves two key components: (i) fostering the growth of nodes to retain previously acquired knowledge, ensuring a rich representation of the environment, and (ii) pruning the least contributing nodes to maintain a more manageable and tractable latent space. Extensive experimental evaluations conducted on three distinct benchmark environments and a real-world molecular environment demonstrate that the proposed SAQN significantly outperforms state-of-the-art counterparts. The results highlight the effectiveness of the self-evolving autoencoder and its collaboration with the Q-Network in tackling sequential decision-making tasks.

Rajdeep Dutta, Qincheng Wang, Ankur Singh et al.

This paper presents a novel RL algorithm, S-REINFORCE, which is designed to generate interpretable policies for dynamic decision-making tasks. The proposed algorithm leverages two types of function approximators, namely Neural Network (NN) and Symbolic Regressor (SR), to produce numerical and symbolic policies, respectively. The NN component learns to generate a numerical probability distribution over the possible actions using a policy gradient, while the SR component captures the functional form that relates the associated states with the action probabilities. The SR-generated policy expressions are then utilized through importance sampling to improve the rewards received during the learning process. We have tested the proposed S-REINFORCE algorithm on various dynamic decision-making problems with low and high dimensional action spaces, and the results demonstrate its effectiveness and impact in achieving interpretable solutions. By leveraging the strengths of both NN and SR, S-REINFORCE produces policies that are not only well-performing but also easy to interpret, making it an ideal choice for real-world applications where transparency and causality are crucial.

Rajdeep Dutta, Chunjiang Qian, Liang Sun et al.

In this paper, we present a novel decentralized controller to drive multiple unmanned aerial vehicles (UAVs) into a symmetric formation of regular polygon shape surrounding a mobile target. The proposed controller works for time-varying information exchange topologies among agents and preserves a network connectivity while steering UAVs into a formation. The proposed nonlinear controller is highly generalized and offers flexibility in achieving the control objective due to the freedom of choosing controller parameters from a range of values. By the virtue of additional tuning parameters, i.e. fractional powers on proportional and derivative difference terms, the nonlinear controller procures a family of UAV trajectories satisfying the same control objective. An appropriate adjustment of the parameters facilitates in generating smooth UAV trajectories without causing abrupt position jumps. The convergence of the closed-loop system is analyzed and established using the Lyapunov approach. Simulation results validate the effectiveness of the proposed controller which outperforms an existing formation controller by driving a team of UAVs elegantly in a target-centric formation. We also present a nonlinear observer to estimate vehicle velocities with the availability of position coordinates and heading angles. Simulation results show that the proposed nonlinear observer results in quick convergence of the estimates to its true values.