Thiagarajan Ramachandran

Aowabin Rahman, Arnab Bhattacharya, Thiagarajan Ramachandran et al.

Search and Rescue (SAR) missions in remote environments often employ autonomous multi-robot systems that learn, plan, and execute a combination of local single-robot control actions, group primitives, and global mission-oriented coordination and collaboration. Often, SAR coordination strategies are manually designed by human experts who can remotely control the multi-robot system and enable semi-autonomous operations. However, in remote environments where connectivity is limited and human intervention is often not possible, decentralized collaboration strategies are needed for fully-autonomous operations. Nevertheless, decentralized coordination may be ineffective in adversarial environments due to sensor noise, actuation faults, or manipulation of inter-agent communication data. In this paper, we propose an algorithmic approach based on adversarial multi-agent reinforcement learning (MARL) that allows robots to efficiently coordinate their strategies in the presence of adversarial inter-agent communications. In our setup, the objective of the multi-robot team is to discover targets strategically in an obstacle-strewn geographical area by minimizing the average time needed to find the targets. It is assumed that the robots have no prior knowledge of the target locations, and they can interact with only a subset of neighboring robots at any time. Based on the centralized training with decentralized execution (CTDE) paradigm in MARL, we utilize a hierarchical meta-learning framework to learn dynamic team-coordination modalities and discover emergent team behavior under complex cooperative-competitive scenarios. The effectiveness of our approach is demonstrated on a collection of prototype grid-world environments with different specifications of benign and adversarial agents, target locations, and agent rewards.

Sai Pushpak Nandanoori, Indrasis Chakraborty, Thiagarajan Ramachandran et al.

The potential of distributed energy resources in providing grid services can be maximized with the recent advancements in demand side control. Effective utilization of this control strategy requires the knowledge of aggregate flexibility of the distributed energy resources (DERs). Recent works have shown that the aggregate flexibility of DERs can be modeled as a virtual battery (VB) whose state evolution is governed by a first order system including self-dissipation. The VB parameters (self-dissipation rate, energy capacity) are obtained by solving an optimization problem which minimizes the tracking performance of the ensemble and the proposed first-order model. For the identified first order model, time-varying power limits are calculated using binary search algorithms. Finally, this proposed framework is demonstrated for different homogeneous and heterogeneous ensembles consisting of air conditioners (ACs) and electric water heaters (EWHs).

Shiva Poudel, Thiagarajan Ramachandran, Orestis Vasios et al.

The increasing use of LLM-based agents to support decision-making and control across diverse domains motivates the need for systematic deconfliction of their proposed actions. We present a deconfliction framework for coordinating multiple agents that formally encapsulate individual applications, each proposing potentially conflicting actions over shared resources. Conflicts are resolved through three deconfliction modes: bilateral negotiation, structured mediation, and procedural (deterministic) deconfliction. We define design principles for large language model-based client agents, including a chain-of-thought style reasoning process, and introduce an iterative weighted-consensus mechanism that does not require the applications themselves to solve optimization problems. The framework is domain agnostic and supports both numeric and non-numeric decisions. Its performance is demonstrated on a power distribution use case with conflicting advanced distribution management system applications for cost optimization and resilience, coordinating diesel generators and battery energy storage systems.

Arnab Bhattacharya, Thiagarajan Ramachandran, Sandeep Banik et al.

Adversary emulation is an offensive exercise that provides a comprehensive assessment of a system's resilience against cyber attacks. However, adversary emulation is typically a manual process, making it costly and hard to deploy in cyber-physical systems (CPS) with complex dynamics, vulnerabilities, and operational uncertainties. In this paper, we develop an automated, domain-aware approach to adversary emulation for CPS. We formulate a Markov Decision Process (MDP) model to determine an optimal attack sequence over a hybrid attack graph with cyber (discrete) and physical (continuous) components and related physical dynamics. We apply model-based and model-free reinforcement learning (RL) methods to solve the discrete-continuous MDP in a tractable fashion. As a baseline, we also develop a greedy attack algorithm and compare it with the RL procedures. We summarize our findings through a numerical study on sensor deception attacks in buildings to compare the performance and solution quality of the proposed algorithms.

Thiagarajan Ramachandran, Andrew Reiman, Sai Pushpak Nandanoori et al.

Low-to-medium voltage distribution networks are experiencing rising levels of distributed energy resources, including renewable generation, along with improved sensing, communication, and automation infrastructure. As such, state estimation methods for distribution systems are becoming increasingly relevant as a means to enable better control strategies that can both leverage the benefits and mitigate the risks associated with high penetration of variable and uncertain distributed generation resources. The primary challenges of this problem include modeling complexities (nonlinear, non-convex power-flow equations), limited availability of sensor measurements, and high penetration of uncertain renewable generation. This paper formulates the distribution system state estimation as a nonlinear, weighted, least squares problem, based on sensor measurements as well as forecast data (both load and generation). We investigate the sensitivity of state estimator accuracy to (load/generation) forecast uncertainties, sensor accuracy, and sensor coverage levels.