Massimo Franceschetti

Mohammad Javad Khojasteh, Pavankumar Tallapragada, Jorge Cortes et al.

Time-triggered and event-triggered control strategies for stabilization of an unstable plant over a rate-limited communication channel subject to unknown, bounded delay are studied and compared. Event triggering carries implicit information, revealing the state of the plant. However, the delay in the communication channel causes information loss, as it makes the state information out of date. There is a critical delay value, when the loss of information due to the communication delay perfectly compensates the implicit information carried by the triggering events. This occurs when the maximum delay equals the inverse of the entropy rate of the plant. In this context, extensions of our previous results for event triggering strategies are presented for vector systems and are compared with the data-rate theorem for time-triggered control, that is extended here to a setting with unknown delay.

Mohammad Javad Khojasteh, Mojtaba Hedayatpour, Jorge Cortes et al.

We present an event-triggered control strategy for stabilizing a scalar, continuous-time, time-invariant, linear system over a digital communication channel having bounded delay, and in the presence of bounded system disturbance. We propose an encoding-decoding scheme, and determine lower bounds on the packet size and on the information transmission rate which are sufficient for stabilization. We show that for small values of the delay, the timing information implicit in the triggering events is enough to stabilize the system with any positive rate. In contrast, when the delay increases beyond a critical threshold, the timing information alone is not enough to stabilize the system and the transmission rate begins to increase. Finally, large values of the delay require transmission rates higher than what prescribed by the classic data-rate theorem. The results are numerically validated using a linearized model of an inverted pendulum.

Anshuka Rangi, Haifeng Xu, Long Tran-Thanh et al.

To understand the security threats to reinforcement learning (RL) algorithms, this paper studies poisoning attacks to manipulate \emph{any} order-optimal learning algorithm towards a targeted policy in episodic RL and examines the potential damage of two natural types of poisoning attacks, i.e., the manipulation of \emph{reward} and \emph{action}. We discover that the effect of attacks crucially depend on whether the rewards are bounded or unbounded. In bounded reward settings, we show that only reward manipulation or only action manipulation cannot guarantee a successful attack. However, by combining reward and action manipulation, the adversary can manipulate any order-optimal learning algorithm to follow any targeted policy with $\tildeΘ(\sqrt{T})$ total attack cost, which is order-optimal, without any knowledge of the underlying MDP. In contrast, in unbounded reward settings, we show that reward manipulation attacks are sufficient for an adversary to successfully manipulate any order-optimal learning algorithm to follow any targeted policy using $\tilde{O}(\sqrt{T})$ amount of contamination. Our results reveal useful insights about what can or cannot be achieved by poisoning attacks, and are set to spur more works on the design of robust RL algorithms.

Mohammad Javad Khojasteh, Mojtaba Hedayatpour, Jorge Cortes et al.

In the same way that subsequent pauses in spoken language are used to convey information, it is also possible to transmit information in communication networks not only by message content, but also with its timing. This paper presents an event-triggering strategy that utilizes timing information by transmitting in a state-dependent fashion. We consider the stabilization of a continuous-time, time-invariant, linear plant over a digital communication channel with bounded delay and subject to bounded plant disturbances and establish two main results. On the one hand, we design an encoding-decoding scheme that guarantees a sufficient information transmission rate for stabilization. On the other hand, we determine a lower bound on the information transmission rate necessary for stabilization by any control policy.

Mohammad Javad Khojasteh, Mojtaba Hedayatpour, Massimo Franceschetti

In the context of event-triggered control, the timing of the triggering events carries information about the state of the system that can be used for stabilization. At each triggering event, not only can information be transmitted by the message content (data payload) but also by its timing. We demonstrate this in the context of stabilization of a laboratory-scale inverted pendulum around its equilibrium point over a digital communication channel with bounded unknown delay. Our event-triggering control strategy encodes timing information by transmitting in a state-dependent fashion and can achieve stabilization using a data payload transmission rate lower than what the data-rate theorem prescribes for classical periodic control policies that do not exploit timing information. Through experimental results, we show that as the delay in the communication channel increases, a higher data payload transmission rate is required to fulfill the proposed event-triggering policy requirements. This confirms the theoretical intuition that a larger delay brings a larger uncertainty about the value of the state at the controller, as less timing information is carried in the communication. In addition, our results also provide a novel encoding-decoding scheme to achieve input-to-state practically stability (ISpS) for nonlinear continuous-time systems under appropriate assumptions.

Mohammad Javad Khojasteh, Massimo Franceschetti, Gireeja Ranade

We consider the problem of stabilizing an undisturbed, scalar, linear system over a "timing" channel, namely a channel where information is communicated through the timestamps of the transmitted symbols. Each symbol transmitted from a sensor to a controller in a closed-loop system is received subject to some to random delay. The sensor can encode messages in the waiting times between successive transmissions and the controller must decode them from the inter-reception times of successive symbols. This set-up is analogous to a telephone system where a transmitter signals a phone call to a receiver through a "ring" and, after the random delay required to establish the connection; the receiver is aware of the "ring" being received. Since there is no data payload exchange between the sensor and the controller, this set-up provides an abstraction for performing event-triggering control with zero-payload rate. We show the following requirement for stabilization: for the state of the system to converge to zero in probability, the timing capacity of the channel should be, essentially, at least as large as the entropy rate of the system. Conversely, in the case the symbol delays are exponentially distributed, we show an "almost" tight sufficient condition using a coding strategy that refines the estimate of the decoded message every time a new symbol is received. Our results generalize previous zero-payload event-triggering control strategies, revealing a fundamental limit in using timing information for stabilization, independent of any transmission strategy.

Xinran Zheng, Leonardo Massai, Massimo Franceschetti et al.

Network centrality is a foundational concept for quantifying the importance of nodes within a network. Many traditional centrality measures--such as degree and betweenness centrality--are purely structural and often overlook the dynamics that unfold across the network. However, the notion of a node's importance is inherently context-dependent and must reflect both the system's dynamics and the specific objectives guiding its operation. Motivated by this perspective, we propose a dynamic, task-aware centrality framework rooted in optimal control theory. By formulating a problem on minimum energy control of average opinion based on Laplacian dynamics and focusing on the variance of terminal state, we introduce a novel centrality measure--termed U-centrality--that quantifies a node's ability to unify the agents' state. We demonstrate that U-centrality interpolates between known measures: it aligns with degree centrality in the short-time horizon and converges to a new centrality over longer time scales which is closely related to current-flow closeness centrality. This work bridges structural and dynamical approaches to centrality, offering a principled, versatile tool for network analysis in dynamic environments.

Anshuka Rangi, Long Tran-Thanh, Haifeng Xu et al.

We study bandit algorithms under data poisoning attacks in a bounded reward setting. We consider a strong attacker model in which the attacker can observe both the selected actions and their corresponding rewards and can contaminate the rewards with additive noise. We show that any bandit algorithm with regret $O(\log T)$ can be forced to suffer a regret $Ω(T)$ with an expected amount of contamination $O(\log T)$. This amount of contamination is also necessary, as we prove that there exists an $O(\log T)$ regret bandit algorithm, specifically the classical UCB, that requires $Ω(\log T)$ amount of contamination to suffer regret $Ω(T)$. To combat such attacks, our second main contribution is to propose verification based mechanisms, which use limited verification to access a limited number of uncontaminated rewards. In particular, for the case of unlimited verifications, we show that with $O(\log T)$ expected number of verifications, a simple modified version of the ETC type bandit algorithm can restore the order optimal $O(\log T)$ regret irrespective of the amount of contamination used by the attacker. We also provide a UCB-like verification scheme, called Secure-UCB, that also enjoys full recovery from any attacks, also with $O(\log T)$ expected number of verifications. To derive a matching lower bound on the number of verifications, we prove that for any order-optimal bandit algorithm, this number of verifications $Ω(\log T)$ is necessary to recover the order-optimal regret. On the other hand, when the number of verifications is bounded above by a budget $B$, we propose a novel algorithm, Secure-BARBAR, which provably achieves $O(\min\{C,T/\sqrt{B} \})$ regret with high probability against weak attackers where $C$ is the total amount of contamination by the attacker, which breaks the known $Ω(C)$ lower bound of the non-verified setting if $C$ is large.

Anshuka Rangi, Massimo Franceschetti, Long Tran-Thanh

In this work, we study sequential choice bandits with feedback. We propose bandit algorithms for a platform that personalizes users' experience to maximize its rewards. For each action directed to a given user, the platform is given a positive reward, which is a non-decreasing function of the action, if this action is below the user's threshold. Users are equipped with a patience budget, and actions that are above the threshold decrease the user's patience. When all patience is lost, the user abandons the platform. The platform attempts to learn the thresholds of the users in order to maximize its rewards, based on two different feedback models describing the information pattern available to the platform at each action. We define a notion of regret by determining the best action to be taken when the platform knows that the user's threshold is in a given interval. We then propose bandit algorithms for the two feedback models and show that upper and lower bounds on the regret are of the order of $\tilde{O}(N^{2/3})$ and $\tildeΩ(N^{2/3})$, respectively, where $N$ is the total number of users. Finally, we show that the waiting time of any user before receiving a personalized experience is uniform in $N$.

Vikas Dhiman, Mohammad Javad Khojasteh, Massimo Franceschetti et al.

This paper focuses on learning a model of system dynamics online while satisfying safety constraints. Our objective is to avoid offline system identification or hand-specified models and allow a system to safely and autonomously estimate and adapt its own model during operation. Given streaming observations of the system state, we use Bayesian learning to obtain a distribution over the system dynamics. Specifically, we propose a new matrix variate Gaussian process (MVGP) regression approach with an efficient covariance factorization to learn the drift and input gain terms of a nonlinear control-affine system. The MVGP distribution is then used to optimize the system behavior and ensure safety with high probability, by specifying control Lyapunov function (CLF) and control barrier function (CBF) chance constraints. We show that a safe control policy can be synthesized for systems with arbitrary relative degree and probabilistic CLF-CBF constraints by solving a second order cone program (SOCP). Finally, we extend our design to a self-triggering formulation, adaptively determining the time at which a new control input needs to be applied in order to guarantee safety.

Anshuka Rangi, Mohammad Javad Khojasteh, Massimo Franceschetti

We study the problem of learning-based attacks in linear systems, where the communication channel between the controller and the plant can be hijacked by a malicious attacker. We assume the attacker learns the dynamics of the system from observations, then overrides the controller's actuation signal, while mimicking legitimate operation by providing fictitious sensor readings to the controller. On the other hand, the controller is on a lookout to detect the presence of the attacker and tries to enhance the detection performance by carefully crafting its control signals. We study the trade-offs between the information acquired by the attacker from observations, the detection capabilities of the controller, and the control cost. Specifically, we provide tight upper and lower bounds on the expected $ε$-deception time, namely the time required by the controller to make a decision regarding the presence of an attacker with confidence at least $(1-ε\log(1/ε))$. We then show a probabilistic lower bound on the time that must be spent by the attacker learning the system, in order for the controller to have a given expected $ε$-deception time. We show that this bound is also order optimal, in the sense that if the attacker satisfies it, then there exists a learning algorithm with the given order expected deception time. Finally, we show a lower bound on the expected energy expenditure required to guarantee detection with confidence at least $1-ε\log(1/ε)$.

Mohammad Javad Khojasteh, Vikas Dhiman, Massimo Franceschetti et al.

This paper focuses on learning a model of system dynamics online while satisfying safety constraints.Our motivation is to avoid offline system identification or hand-specified dynamics models and allowa system to safely and autonomously estimate and adapt its own model during online operation.Given streaming observations of the system state, we use Bayesian learning to obtain a distributionover the system dynamics. In turn, the distribution is used to optimize the system behavior andensure safety with high probability, by specifying a chance constraint over a control barrier function.

Hamed Omidvar, Vahideh Akhlaghi, Massimo Franceschetti et al.

Motivated by the necessity for parameter efficiency in distributed machine learning and AI-enabled edge devices, we provide a general and easy to implement method for significantly reducing the number of parameters of Convolutional Neural Networks (CNNs), during both the training and inference phases. We introduce a simple auxiliary neural network which can generate the convolutional filters of any CNN architecture from a low dimensional latent space. This auxiliary neural network, which we call "Convolutional Slice Generator" (CSG), is unique to the network and provides the association between its convolutional layers. During the training of the CNN, instead of training the filters of the convolutional layers, only the parameters of the CSG and their corresponding "code vectors" are trained. This results in a significant reduction of the number of parameters due to the fact that the CNN can be fully represented using only the parameters of the CSG, the code vectors, the fully connected layers, and the architecture of the CNN. We evaluate our approach by applying it to ResNet and DenseNet models when trained on CIFAR-10 and ImageNet datasets. While reducing the number of parameters by $\approx 2 \times$ on average, the accuracies of these networks remain within 1$\%$ of their original counterparts and in some cases there is an increase in the accuracy.

Anshuka Rangi, Massimo Franceschetti

We study online learning when partial feedback information is provided following every action of the learning process, and the learner incurs switching costs for changing his actions. In this setting, the feedback information system can be represented by a graph, and previous works studied the expected regret of the learner in the case of a clique (Expert setup), or disconnected single loops (Multi-Armed Bandits (MAB)). This work provides a lower bound on the expected regret in the Partial Information (PI) setting, namely for general feedback graphs --excluding the clique. Additionally, it shows that all algorithms that are optimal without switching costs are necessarily sub-optimal in the presence of switching costs, which motivates the need to design new algorithms. We propose two new algorithms: Threshold Based EXP3 and EXP3. SC. For the two special cases of symmetric PI setting and MAB, the expected regret of both of these algorithms is order optimal in the duration of the learning process. Additionally, Threshold Based EXP3 is order optimal in the switching cost, whereas EXP3. SC is not. Finally, empirical evaluations show that Threshold Based EXP3 outperforms the previously proposed order-optimal algorithms EXP3 SET in the presence of switching costs, and Batch EXP3 in the MAB setting with switching costs.

Anshuka Rangi, Massimo Franceschetti, Long Tran-Thanh

This paper investigates the adversarial Bandits with Knapsack (BwK) online learning problem, where a player repeatedly chooses to perform an action, pays the corresponding cost, and receives a reward associated with the action. The player is constrained by the maximum budget $B$ that can be spent to perform actions, and the rewards and the costs of the actions are assigned by an adversary. This problem has only been studied in the restricted setting where the reward of an action is greater than the cost of the action, while we provide a solution in the general setting. Namely, we propose EXP3.BwK, a novel algorithm that achieves order optimal regret. We also propose EXP3++.BwK, which is order optimal in the adversarial BwK setup, and incurs an almost optimal expected regret with an additional factor of $\log(B)$ in the stochastic BwK setup. Finally, we investigate the case of having large costs for the actions (i.e., they are comparable to the budget size $B$), and show that for the adversarial setting, achievable regret bounds can be significantly worse, compared to the case of having costs bounded by a constant, which is a common assumption within the BwK literature.

Mohammad Javad Khojasteh, Anatoly Khina, Massimo Franceschetti et al.

We introduce the problem of learning-based attacks in a simple abstraction of cyber-physical systems---the case of a discrete-time, linear, time-invariant plant that may be subject to an attack that overrides the sensor readings and the controller actions. The attacker attempts to learn the dynamics of the plant and subsequently override the controller's actuation signal, to destroy the plant without being detected. The attacker can feed fictitious sensor readings to the controller using its estimate of the plant dynamics and mimic the legitimate plant operation. The controller, on the other hand, is constantly on the lookout for an attack; once the controller detects an attack, it immediately shuts the plant off. In the case of scalar plants, we derive an upper bound on the attacker's deception probability for any measurable control policy when the attacker uses an arbitrary learning algorithm to estimate the system dynamics. We then derive lower bounds for the attacker's deception probability for both scalar and vector plants by assuming a specific authentication test that inspects the empirical variance of the system disturbance. We also show how the controller can improve the security of the system by superimposing a carefully crafted privacy-enhancing signal on top of the "nominal control policy." Finally, for nonlinear scalar dynamics that belong to the Reproducing Kernel Hilbert Space (RKHS), we investigate the performance of attacks based on nonlinear Gaussian-processes (GP) learning algorithms.

Anshuka Rangi, Massimo Franceschetti, Stefano Marano

We study "active" decision making over sensor networks where the sensors' sequential probing actions are actively chosen by continuously learning from past observations. We consider two network settings: with and without central coordination. In the first case, the network nodes interact with each other through a central entity, which plays the role of a fusion center. In the second case, the network nodes interact in a fully distributed fashion. In both of these scenarios, we propose sequential and adaptive hypothesis tests extending the classic Chernoff test. We compare the performance of the proposed tests to the optimal sequential test. In the presence of a fusion center, our test achieves the same asymptotic optimality of the Chernoff test, minimizing the risk, expressed by the expected cost required to reach a decision plus the expected cost of making a wrong decision, when the observation cost per unit time tends to zero. The test is also asymptotically optimal in the higher moments of the time required to reach a decision. Additionally, the test is parsimonious in terms of communications, and the expected number of channel uses per network node tends to a small constant. In the distributed setup, our test achieves the same asymptotic optimality of Chernoff's test, up to a multiplicative constant in terms of both risk and the higher moments of the decision time. Additionally, the test is parsimonious in terms of communications in comparison to state-of-the-art schemes proposed in the literature. The analysis of these tests is also extended to account for message quantization and communication over channels with random erasures.

Pavankumar Tallapragada, Massimo Franceschetti, Jorge Cortes

This paper deals with the stabilization of linear systems with process noise under packet drops between the sensor and the controller. Our aim is to ensure exponential convergence of the second moment of the plant state to a given bound in finite time. Motivated by considerations about the efficient use of the available resources, we adopt an event-triggering approach to design the transmission policy. In our design, the sensor's decision to transmit or not the state to the controller is based on an online evaluation of the future satisfaction of the control objective. The resulting event-triggering policy is hence specifically tailored to the control objective. We formally establish that the proposed event-triggering policy meets the desired objective and quantify its efficiency by providing an upper bound on the fraction of expected number of transmissions in an infinite time interval. Simulations for scalar and vector systems illustrate the results.