Bo Wu

Mohamadreza Ahmadi, Bo Wu, Hai Lin et al.

Privacy is an increasing concern in cyber-physical systems that operates over a shared network. In this paper, we propose a method for privacy verification of cyber- physical systems modeled by Markov decision processes (MDPs) and partially-observable Markov decision processes (POMDPs) based on barrier certificates. To this end, we consider an opacity-based notion of privacy, which is characterized by the beliefs in system states. We show that the belief update equations can be represented as discrete-time switched systems, for which we propose a set of conditions for privacy verification in terms of barrier certificates. We further demonstrate that, for MDPs and for POMDPs, privacy verification can be computationally implemented by solving a set of semi-definite programs and sum-of-squares programs, respectively. The method is illustrated by an application to privacy verification of an inventory management system.

Zhiyu Liu, Bo Wu, Jin Dai et al.

Control and communication are often tightly coupled in motion planning of networked mobile robots, due to the fact that robotic motions will affect the overall communication quality, and the quality of service (QoS) of the communication among the robots will in turn affect their coordination performance. In this paper, we propose a control theoretical motion planning framework for a team of networked mobile robots in order to accomplish high-level spatial and temporal motion objectives while optimizing communication QoS. Desired motion specifications are formulated as Signal Temporal Logic (STL), whereas the communication performances to be optimized are captured by recently proposed Spatial Temporal Reach and Escape Logic (STREL) formulas. Both the STL and STREL specifications are encoded as mixed integer linear constraints posed on the system and/or environment state variables of the mobile robot network, where satisfactory control strategies can be computed by exploiting a distributed model predictive control (MPC) approach. To the best of the authors' knowledge, we are the first to study controller synthesis for STREL specifications. A two-layer hierarchical MPC procedure is proposed to efficiently solve the problem, whose soundness and completeness are formally ensured. The effectiveness of the proposed framework is validated by simulation examples.

Mohamadreza Ahmadi, Bo Wu, Yuxin Chen et al.

Machine teaching can be viewed as optimal control for learning. Given a learner's model, machine teaching aims to determine the optimal training data to steer the learner towards a target hypothesis. In this paper, we are interested in providing assurances for machine teaching algorithms using control theory. In particular, we study a well-established learner's model in the machine teaching literature that is captured by the local preference over a version space. We interpret the problem of teaching a preference-based learner as solving a partially observable Markov decision process (POMDP). We then show that the POMDP formulation can be cast as a special hybrid system, i.e., a discrete-time switched system. Subsequently, we use barrier certificates to verify set-theoric properties of this special hybrid system. We show how the computation of the barrier certificate can be decomposed and numerically implemented as the solution to a sum-of-squares (SOS) program. For illustration, we show how the proposed framework based on control theory can be used to verify the teaching performance of two well-known machine teaching methods.

Bo Wu, Xiaobin Zhang, Hai Lin

This paper considers the permissive supervisor synthesis for probabilistic systems modeled as Markov Decision Processes (MDP). Such systems are prevalent in power grids, transportation networks, communication networks and robotics. Unlike centralized planning and optimization based planning, we propose a novel supervisor synthesis framework based on learning and compositional model checking to generate permissive local supervisors in a distributed manner. With the recent advance in assume-guarantee reasoning verification for probabilistic systems, building the composed system can be avoided to alleviate the state space explosion and our framework learn the supervisors iteratively based on the counterexamples from verification. Our approach is guaranteed to terminate in finite steps and to be correct.

Xiaobin Zhang, Bo Wu, Hai Lin

Partially Observable Markov Decision Process (POMDP) is widely used to model probabilistic behavior for complex systems. Compared with MDPs, POMDP models a system more accurate but solving a POMDP generally takes exponential time in the size of its state space. This makes the formal verification and synthesis problems much more challenging for POMDPs, especially when multiple system components are involved. As a promising technique to reduce the verification complexity, the abstraction method tries to find an abstract system with a smaller state space but preserves enough properties for the verification purpose. While abstraction based verification has been explored extensively for MDPs, in this paper, we present the first result of POMDP abstraction and its refinement techniques. The main idea follows the counterexample-guided abstraction refinement (CEGAR) framework. Starting with a coarse guess for the POMDP abstraction, we iteratively use counterexamples from formal verification to refine the abstraction until the abstract system can be used to infer the verification result for the original POMDP. Our main contributions have two folds: 1) we propose a novel abstract system model for POMDP and a new simulation relation to capture the partial observability then prove the preservation on a fragment of Probabilistic Computation Tree Logic (PCTL); 2) to find a proper abstract system that can prove or disprove the satisfaction relation on the concrete POMDP, we develop a novel refinement algorithm. Our work leads to a sound and complete CEGAR framework for POMDP.

David Pujol-Perich, José Suárez-Varela, Miquel Ferriol et al.

Recent years have seen the vast potential of Graph Neural Networks (GNN) in many fields where data is structured as graphs (e.g., chemistry, recommender systems). In particular, GNNs are becoming increasingly popular in the field of networking, as graphs are intrinsically present at many levels (e.g., topology, routing). The main novelty of GNNs is their ability to generalize to other networks unseen during training, which is an essential feature for developing practical Machine Learning (ML) solutions for networking. However, implementing a functional GNN prototype is currently a cumbersome task that requires strong skills in neural network programming. This poses an important barrier to network engineers that often do not have the necessary ML expertise. In this article, we present IGNNITION, a novel open-source framework that enables fast prototyping of GNNs for networking systems. IGNNITION is based on an intuitive high-level abstraction that hides the complexity behind GNNs, while still offering great flexibility to build custom GNN architectures. To showcase the versatility and performance of this framework, we implement two state-of-the-art GNN models applied to different networking use cases. Our results show that the GNN models produced by IGNNITION are equivalent in terms of accuracy and performance to their native implementations in TensorFlow.

José Suárez-Varela, Miquel Ferriol-Galmés, Albert López et al.

During the last decade, Machine Learning (ML) has increasingly become a hot topic in the field of Computer Networks and is expected to be gradually adopted for a plethora of control, monitoring and management tasks in real-world deployments. This poses the need to count on new generations of students, researchers and practitioners with a solid background in ML applied to networks. During 2020, the International Telecommunication Union (ITU) has organized the "ITU AI/ML in 5G challenge'', an open global competition that has introduced to a broad audience some of the current main challenges in ML for networks. This large-scale initiative has gathered 23 different challenges proposed by network operators, equipment manufacturers and academia, and has attracted a total of 1300+ participants from 60+ countries. This paper narrates our experience organizing one of the proposed challenges: the "Graph Neural Networking Challenge 2020''. We describe the problem presented to participants, the tools and resources provided, some organization aspects and participation statistics, an outline of the top-3 awarded solutions, and a summary with some lessons learned during all this journey. As a result, this challenge leaves a curated set of educational resources openly available to anyone interested in the topic.

Parham Gohari, Bo Chen, Bo Wu et al.

Kickstarting deep reinforcement learning algorithms facilitate a teacher-student relationship among the agents and allow for a well-performing teacher to share demonstrations with a student to expedite the student's training. However, despite the known benefits, the demonstrations may contain sensitive information about the teacher's training data and existing kickstarting methods do not take any measures to protect it. Therefore, we use the framework of differential privacy to develop a mechanism that securely shares the teacher's demonstrations with the student. The mechanism allows for the teacher to decide upon the accuracy of its demonstrations with respect to the privacy budget that it consumes, thereby granting the teacher full control over its data privacy. We then develop a kickstarted deep reinforcement learning algorithm for the student that is privacy-aware because we calibrate its objective with the parameters of the teacher's privacy mechanism. The privacy-aware design of the algorithm makes it possible to kickstart the student's learning despite the perturbations induced by the privacy mechanism. From numerical experiments, we highlight three empirical results: (i) the algorithm succeeds in expediting the student's learning, (ii) the student converges to a performance level that was not possible without the demonstrations, and (iii) the student maintains its enhanced performance even after the teacher stops sharing useful demonstrations due to its privacy budget constraints.

Zhe Xu, Federico M. Zegers, Bo Wu et al.

This paper investigates the controller synthesis problem for a multi-agent system (MAS) with intermittent communication. We adopt a relay-explorer scheme, where a mobile relay agent with absolute position sensors switches among a set of explorers with relative position sensors to provide intermittent state information. We model the MAS as a switched system where the explorers' dynamics can be either fully-actuated or underactuated. The objective of the explorers is to reach approximate consensus to a predetermined goal region. To guarantee the stability of the switched system and the approximate consensus of the explorers, we derive maximum dwell-time conditions to constrain the length of time each explorer goes without state feedback (from the relay agent). Furthermore, the relay agent needs to satisfy practical constraints such as charging its battery and staying in specific regions of interest. Both the maximum dwell-time conditions and these practical constraints can be expressed by metric temporal logic (MTL) specifications. We iteratively compute the optimal control inputs for the relay agent to satisfy the MTL specifications, while guaranteeing stability and approximate consensus of the explorers. We implement the proposed method on a case study with the CoppeliaSim robot simulator.

Cyrus Neary, Zhe Xu, Bo Wu et al.

In cooperative multi-agent reinforcement learning, a collection of agents learns to interact in a shared environment to achieve a common goal. We propose the use of reward machines (RM) -- Mealy machines used as structured representations of reward functions -- to encode the team's task. The proposed novel interpretation of RMs in the multi-agent setting explicitly encodes required teammate interdependencies, allowing the team-level task to be decomposed into sub-tasks for individual agents. We define such a notion of RM decomposition and present algorithmically verifiable conditions guaranteeing that distributed completion of the sub-tasks leads to team behavior accomplishing the original task. This framework for task decomposition provides a natural approach to decentralized learning: agents may learn to accomplish their sub-tasks while observing only their local state and abstracted representations of their teammates. We accordingly propose a decentralized q-learning algorithm. Furthermore, in the case of undiscounted rewards, we use local value functions to derive lower and upper bounds for the global value function corresponding to the team task. Experimental results in three discrete settings exemplify the effectiveness of the proposed RM decomposition approach, which converges to a successful team policy an order of magnitude faster than a centralized learner and significantly outperforms hierarchical and independent q-learning approaches.

Yuqian Jiang, Sudarshanan Bharadwaj, Bo Wu et al.

In continuing tasks, average-reward reinforcement learning may be a more appropriate problem formulation than the more common discounted reward formulation. As usual, learning an optimal policy in this setting typically requires a large amount of training experiences. Reward shaping is a common approach for incorporating domain knowledge into reinforcement learning in order to speed up convergence to an optimal policy. However, to the best of our knowledge, the theoretical properties of reward shaping have thus far only been established in the discounted setting. This paper presents the first reward shaping framework for average-reward learning and proves that, under standard assumptions, the optimal policy under the original reward function can be recovered. In order to avoid the need for manual construction of the shaping function, we introduce a method for utilizing domain knowledge expressed as a temporal logic formula. The formula is automatically translated to a shaping function that provides additional reward throughout the learning process. We evaluate the proposed method on three continuing tasks. In all cases, shaping speeds up the average-reward learning rate without any reduction in the performance of the learned policy compared to relevant baselines.

Zhe Xu, Bo Wu, Aditya Ojha et al.

Despite the fact that deep reinforcement learning (RL) has surpassed human-level performances in various tasks, it still has several fundamental challenges. First, most RL methods require intensive data from the exploration of the environment to achieve satisfactory performance. Second, the use of neural networks in RL renders it hard to interpret the internals of the system in a way that humans can understand. To address these two challenges, we propose a framework that enables an RL agent to reason over its exploration process and distill high-level knowledge for effectively guiding its future explorations. Specifically, we propose a novel RL algorithm that learns high-level knowledge in the form of a finite reward automaton by using the L* learning algorithm. We prove that in episodic RL, a finite reward automaton can express any non-Markovian bounded reward functions with finitely many reward values and approximate any non-Markovian bounded reward function (with infinitely many reward values) with arbitrary precision. We also provide a lower bound for the episode length such that the proposed RL approach almost surely converges to an optimal policy in the limit. We test this approach on two RL environments with non-Markovian reward functions, choosing a variety of tasks with increasing complexity for each environment. We compare our algorithm with the state-of-the-art RL algorithms for non-Markovian reward functions, such as Joint Inference of Reward machines and Policies for RL (JIRP), Learning Reward Machine (LRM), and Proximal Policy Optimization (PPO2). Our results show that our algorithm converges to an optimal policy faster than other baseline methods.

Suda Bharadwaj, Louis Ly, Bo Wu et al.

This paper studies a two-player game with a quantitative surveillance requirement on an adversarial target moving in a discrete state space and a secondary objective to maximize short-term visibility of the environment. We impose the surveillance requirement as a temporal logic constraint.We then use a greedy approach to determine vantage points that optimize a notion of information gain, namely, the number of newly-seen states. By using a convolutional neural network trained on a class of environments, we can efficiently approximate the information gain at each potential vantage point.Subsequent vantage points are chosen such that moving to that location will not jeopardize the surveillance requirement, regardless of any future action chosen by the target. Our method combines guarantees of correctness from formal methods with the scalability of machine learning to provide an efficient approach for surveillance-constrained visibility optimization.

Parham Gohari, Bo Wu, Matthew Hale et al.

As members of a network share more information with each other and network providers, sensitive data leakage raises privacy concerns. To address this need for a class of problems, we introduce a novel mechanism that privatizes vectors belonging to the unit simplex. Such vectors can be seen in many applications, such as privatizing a decision-making policy in a Markov decision process. We use differential privacy as the underlying mathematical framework for these developments. The introduced mechanism is a probabilistic mapping that maps a vector within the unit simplex to the same domain according to a Dirichlet distribution. We find the mechanism well-suited for inputs within the unit simplex because it always returns a privatized output that is also in the unit simplex. Therefore, no further projection back onto the unit simplex is required. We verify the privacy guarantees of the mechanism for two cases, namely, identity queries and average queries. In the former case, we derive expressions for the differential privacy level of privatizing a single vector within the unit simplex. In the latter case, we study the mechanism for privatizing the average of a collection of vectors, each of which is in the unit simplex. We establish a trade-off between the strength of privacy and the variance of the mechanism output, and we introduce a parameter to balance the trade-off between them. Numerical results illustrate these developments.

Zhe Xu, Federico M. Zegers, Bo Wu et al.

This paper develops a controller synthesis approach for a multi-agent system (MAS) with intermittent communication. We adopt a leader-follower scheme, where a mobile leader with absolute position sensors switches among a set of followers without absolute position sensors to provide each follower with intermittent state information.We model the MAS as a switched system. The followers are to asymptotically reach a predetermined consensus state. To guarantee the stability of the switched system and the consensus of the followers, we derive maximum and minimal dwell-time conditions to constrain the intervals between consecutive time instants at which the leader should provide state information to the same follower. Furthermore, the leader needs to satisfy practical constraints such as charging its battery and staying in specific regions of interest. Both the maximum and minimum dwell-time conditions and these practical constraints can be expressed by metric temporal logic (MTL) specifications. We iteratively compute the optimal control inputs such that the leader satisfies the MTL specifications, while guaranteeing stability and consensus of the followers. We implement the proposed method on a case study with three mobile robots as the followers and one quadrotor as the leader.

Zhe Xu, Ivan Gavran, Yousef Ahmad et al.

Incorporating high-level knowledge is an effective way to expedite reinforcement learning (RL), especially for complex tasks with sparse rewards. We investigate an RL problem where the high-level knowledge is in the form of reward machines, i.e., a type of Mealy machine that encodes the reward functions. We focus on a setting in which this knowledge is a priori not available to the learning agent. We develop an iterative algorithm that performs joint inference of reward machines and policies for RL (more specifically, q-learning). In each iteration, the algorithm maintains a hypothesis reward machine and a sample of RL episodes. It derives q-functions from the current hypothesis reward machine, and performs RL to update the q-functions. While performing RL, the algorithm updates the sample by adding RL episodes along which the obtained rewards are inconsistent with the rewards based on the current hypothesis reward machine. In the next iteration, the algorithm infers a new hypothesis reward machine from the updated sample. Based on an equivalence relationship we defined between states of reward machines, we transfer the q-functions between the hypothesis reward machines in consecutive iterations. We prove that the proposed algorithm converges almost surely to an optimal policy in the limit if a minimal reward machine can be inferred and the maximal length of each RL episode is sufficiently long. The experiments show that learning high-level knowledge in the form of reward machines can lead to fast convergence to optimal policies in RL, while standard RL methods such as q-learning and hierarchical RL methods fail to converge to optimal policies after a substantial number of training steps in many tasks.

Bo Wu, Murat Cubuktepe, Suda Bharadwaj et al.

Deception plays a key role in adversarial or strategic interactions for the purpose of self-defence and survival. This paper introduces a general framework and solution to address deception. Most existing approaches for deception consider obfuscating crucial information to rational adversaries with abundant memory and computation resources. In this paper, we consider deceiving adversaries with bounded rationality and in terms of expected rewards. This problem is commonly encountered in many applications especially involving human adversaries. Leveraging the cognitive bias of humans in reward evaluation under stochastic outcomes, we introduce a framework to optimally assign resources of a limited quantity to optimally defend against human adversaries. Modeling such cognitive biases follows the so-called prospect theory from behavioral psychology literature. Then we formulate the resource allocation problem as a signomial program to minimize the defender's cost in an environment modeled as a Markov decision process. We use police patrol hour assignment as an illustrative example and provide detailed simulation results based on real-world data.

Mohamadreza Ahmadi, Nils Jansen, Bo Wu et al.

Partially observable Markov decision processes (POMDPs) provide a modeling framework for a variety of sequential decision making under uncertainty scenarios in artificial intelligence (AI). Since the states are not directly observable in a POMDP, decision making has to be performed based on the output of a Bayesian filter (continuous beliefs). Hence, POMDPs are often computationally intractable to solve exactly and researchers resort to approximate methods often using discretizations of the continuous belief space. These approximate solutions are, however, prone to discretization errors, which has made POMDPs ineffective in applications, wherein guarantees for safety, optimality, or performance are required. To overcome the complexity challenge of POMDPs, we apply notions from control theory. The goal is to determine the reachable belief space of a POMDP, that is, the set of all possible evolutions given an initial belief distribution over the states and a set of actions and observations. We begin by casting the problem of analyzing a POMDP into analyzing the behavior of a discrete-time switched system. For estimating the reachable belief space, we find over-approximations in terms of sub-level sets of Lyapunov functions. Furthermore, in order to verify safety and optimality requirements of a given POMDP, we formulate a barrier certificate theorem, wherein we show that if there exists a barrier certificate satisfying a set of inequalities along with the belief update equation of the POMDP, the safety and optimality properties are guaranteed to hold. In both cases, we show how the calculations can be decomposed into smaller problems that can be solved in parallel. The conditions we formulate can be computationally implemented as a set of sum-of-squares programs. We illustrate the applicability of our method by addressing two problems in active ad scheduling and machine teaching.

Wei Zheng, Bo Wu, Hai Lin

Recent years have seen human robot collaboration (HRC) quickly emerged as a hot research area at the intersection of control, robotics, and psychology. While most of the existing work in HRC focused on either low-level human-aware motion planning or HRC interface design, we are particularly interested in a formal design of HRC with respect to high-level complex missions, where it is of critical importance to obtain an accurate and meanwhile tractable human model. Instead of assuming the human model is given, we ask whether it is reasonable to learn human models from observed perception data, such as the gesture, eye movements, head motions of the human in concern. As our initial step, we adopt a partially observable Markov decision process (POMDP) model in this work as mounting evidences have suggested Markovian properties of human behaviors from psychology studies. In addition, POMDP provides a general modeling framework for sequential decision making where states are hidden and actions have stochastic outcomes. Distinct from the majority of POMDP model learning literature, we do not assume that the state, the transition structure or the bound of the number of states in POMDP model is given. Instead, we use a Bayesian non-parametric learning approach to decide the potential human states from data. Then we adopt an approach inspired by probably approximately correct (PAC) learning to obtain not only an estimation of the transition probability but also a confidence interval associated to the estimation. Then, the performance of applying the control policy derived from the estimated model is guaranteed to be sufficiently close to the true model. Finally, data collected from a driver-assistance test-bed are used to train the model, which illustrates the effectiveness of the proposed learning method.

Bo Wu, Hai Lin

Privacy is a crucial concern in many systems in addition to their given tasks. We consider a new notion of privacy based on beliefs of the system states, which is closely related to opacity in discrete event systems. To guarantee the privacy requirement, we propose to abstract the belief space whose dynamics is shown to be mixed monotone where efficient abstraction algorithm exists. Based on the abstraction, we propose two different approaches to synthesize controllers of the system to preserve privacy with an illustrative example.

Bo Wu, Jin Dai, Hai Lin

Opacity is a confidentiality property that characterizes the non-disclosure of specified secret information of a system to an outside observer. In this paper, we consider the enforcement of opacity within the discrete-event system formalism in the presence of multiple intruders. We study two cases, one without coordination among the intruders and the other with coordination. We propose appropriate notions of opacity corresponding to the two cases, respectively, and propose enforcement mechanisms for these opacity properties based on the implementation of insertion functions, which manipulates the output of the system by inserting fictitious observable events whenever necessary. The insertion mechanism is adapted to the decentralized framework to enforce opacity when no coordination exists. Furthermore, we present a coordination and refinement procedure to synthesize appropriate insertion functions to enforce opacity when intruders may coordinate with each other by following an intersection-based coordination protocol. The effectiveness of the proposed opacity-enforcement approaches is validated through illustrative examples.

Bo Wu, Zhiyu Liu, Hai Lin

Opacity is a property that characterizes the system's capability to keep its "secret" from being inferred by an intruder that partially observes the system's behavior. In this paper, we are concerned with enhancing the opacity using insertion functions, while at the same time, enforcing the task specification in a parametric stochastic discrete event system. We first obtain the parametric Markov decision process that encodes all the possible insertions. Based on which, we convert this parameter and insertion function co-synthesis problem into a nonlinear program. We prove that if the output of this program satisfies all the constraints, it will be a valid solution to our problem. Therefore, the security and the capability of enforcing the task specification can be simultaneously guaranteed.

Bo Wu, Bin Hu, Hai Lin

This paper considers an optimal task allocation problem for human robot collaboration in human robot systems with persistent tasks. Such human robot systems consist of human operators and intelligent robots collaborating with each other to accomplish complex tasks that cannot be done by either part alone. The system objective is to maximize the probability of successfully executing persistent tasks that are formulated as linear temporal logic specifications and minimize the average cost between consecutive visits of a particular proposition. This paper proposes to model the human robot collaboration under a framework with the composition of multiple Markov Decision Process (MDP) with possibly unknown transition probabilities, which characterizes how human cognitive states, such as human trust and fatigue, stochastically change with the robot performance. Under the unknown MDP models, an algorithm is developed to learn the model and obtain an optimal task allocation policy that minimizes the expected average cost for each task cycle and maximizes the probability of satisfying linear temporal logic constraints. Moreover, this paper shows that the difference between the optimal policy based on the learned model and that based on the underlying ground truth model can be bounded by arbitrarily small constant and large confidence level with sufficient samples. The case study of an assembly process demonstrates the effectiveness and benefits of our proposed learning based human robot collaboration.

Zhiyu Liu, Bo Wu, Jin Dai et al.

In future intelligent transportation systems, networked vehicles coordinate with each other to achieve safe operations based on an assumption that communications among vehicles and infrastructure are reliable. Traditional methods usually deal with the design of control systems and communication networks in a separated manner. However, control and communication systems are tightly coupled as the motions of vehicles will affect the overall communication quality. Hence, we are motivated to study the co-design of both control and communication systems. In particular, we propose a control theoretical framework for distributed motion planning for multi-agent systems which satisfies complex and high-level spatial and temporal specifications while accounting for communication quality at the same time. Towards this end, desired motion specifications and communication performances are formulated as signal temporal logic (STL) and spatial-temporal logic (SpaTeL) formulas, respectively. The specifications are encoded as constraints on system and environment state variables of mixed integer linear programs (MILP), and upon which control strategies satisfying both STL and SpaTeL specifications are generated for each agent by employing a distributed model predictive control (MPC) framework. Effectiveness of the proposed framework is validated by a simulation of distributed communication-aware motion planning for multi-agent systems.

Xiaobin Zhang, Bo Wu, Hai Lin

As a general and thus popular model for autonomous systems, partially observable Markov decision process (POMDP) can capture uncertainties from different sources like sensing noises, actuation errors, and uncertain environments. However, its comprehensiveness makes the planning and control in POMDP difficult. Traditional POMDP planning problems target to find the optimal policy to maximize the expectation of accumulated rewards. But for safety critical applications, guarantees of system performance described by formal specifications are desired, which motivates us to consider formal methods to synthesize supervisor for POMDP. With system specifications given by Probabilistic Computation Tree Logic (PCTL), we propose a supervisory control framework with a type of deterministic finite automata (DFA), za-DFA, as the controller form. While the existing work mainly relies on optimization techniques to learn fixed-size finite state controllers (FSCs), we develop an $L^*$ learning based algorithm to determine both space and transitions of za-DFA. Membership queries and different oracles for conjectures are defined. The learning algorithm is sound and complete. An example is given in detailed steps to illustrate the supervisor synthesis algorithm.

Zhiyu Liu, Jin Dai, Bo Wu et al.

We propose a mathematical framework for synthesizing motion plans for multi-agent systems that fulfill complex, high-level and formal local specifications in the presence of inter-agent communication. The proposed synthesis framework consists of desired motion specifications in temporal logic (STL) formulas and a local motion controller that ensures the underlying agent not only to accomplish the local specifications but also to avoid collisions with other agents or possible obstacles, while maintaining an optimized communication quality of service (QoS) among the agents. Utilizing a Gaussian fading model for wireless communication channels, the framework synthesizes the desired motion controller by solving a joint optimization problem on motion planning and wireless communication, in which both the STL specifications and the wireless communication conditions are encoded as mixed integer-linear constraints on the variables of the agents' dynamical states and communication channel status. The overall framework is demonstrated by a case study of communication-aware multi-robot motion planning and the effectiveness of the framework is validated by simulation results.

Rafael Rodrigues da Silva, Bo Wu, Jin Dai et al.

We propose a hierarchical design framework to automatically synthesize coordination schemes and control policies for cooperative multi-agent systems to fulfill formal performance requirements, by associating a bottom-up reactive motion controller with a top-down mission plan. On one hand, starting from a global mission that is specified as a regular language over all the agents' mission capabilities, a mission planning layer sits on the top of the proposed framework, decomposing the global mission into local tasks that are in consistency with each agent's individual capabilities, and compositionally justifying whether the achievement of local tasks implies the satisfaction of the global mission via an assume-guarantee paradigm. On the other hand, bottom-up motion plans associated with each agent are synthesized corresponding to the obtained local missions by composing basic motion primitives, which are verified safe by differential dynamic logic (d$\mathcal{L}$), through a Satisfiability Modulo Theories (SMT) solver that searches feasible solutions in face of constraints imposed by local task requirements and the environment description. It is shown that the proposed framework can handle dynamical environments as the motion primitives possess reactive features, making the motion plans adaptive to local environmental changes. Furthermore, on-line mission reconfiguration can be triggered by the motion planning layer once no feasible solutions can be found through the SMT solver. The effectiveness of the overall design framework is validated by an automated warehouse case study.

Rafael Rodrigues da Silva, Bo Wu, Hai Lin

Integrated Task and Motion Planning (ITMP) for mobile robots in a dynamic environment with moving obstacles is a challenging research question and attracts more and more attentions recently. Most existing methods either restrict to static environments or lack performance guarantees. This motivates us to investigate the ITMP problem using formal methods and propose a bottom-up compositional design approach called CoSMoP (Composition of Safe Motion Primitives). Our basic idea is to synthesize a global motion plan through composing simple local moves and actions, and to achieve its performance guarantee through modular and incremental verifications. The design consists of two steps. First, basic motion primitives are designed and verified locally. Then, a global motion path is built upon these certified motion primitives by concatenating them together. In particular, we model the motion primitives as hybrid automata and verify their safety through formulating as Differential Dynamic Logic (d$\mathcal{L}$). Furthermore, these proven safe motion primitives are composed based on an encoding to Satisfiability Modulo Theories (SMT) that takes into account the geometric constraints. Since d$\mathcal{L}$ allows compositional verification, the sequential composition of the safe motion primitives also preserves safety properties. Therefore, the CoSMoP generates correct plans for given task specifications that are formally proven safe even for moving obstacles. Illustrative examples are presented to show the effectiveness of the methods.