Georgios Bakirtzis

Minghui Sun, Zhaoyang Chen, Georgios Bakirtzis et al.

Designing systems is typically uncertain and ambiguous at early stages. Set-based design supports alternative exploration and gradual uncertainty reduction during the early lifecycle, making it practical for complex systems design. In parallel, the functional requirements decomposition helps to advance the design incrementally. However, current literature on set-based design lacks formal guidance in how to decompose functional requirements. To bridge this gap, we introduce a four-step method to decompose functional requirements for set-based design hierarchically. We systematically define, reason, and narrow the sets, breaking down the functional requirements into formal sub-requirements. This method allows parallel abstraction, ensuring the resulting system satisfies the top-level functional requirements.

Tichakorn Wongpiromsarn, Mahsa Ghasemi, Murat Cubuktepe et al.

Formal methods refer to rigorous, mathematical approaches to system development and have played a key role in establishing the correctness of safety-critical systems. The main building blocks of formal methods are models and specifications, which are analogous to behaviors and requirements in system design and give us the means to verify and synthesize system behaviors with formal guarantees. This monograph provides a survey of the current state of the art on applications of formal methods in the autonomous systems domain. We consider correct-by-construction synthesis under various formulations, including closed systems, reactive, and probabilistic settings. Beyond synthesizing systems in known environments, we address the concept of uncertainty and bound the behavior of systems that employ learning using formal methods. Further, we examine the synthesis of systems with monitoring, a mitigation technique for ensuring that once a system deviates from expected behavior, it knows a way of returning to normalcy. We also show how to overcome some limitations of formal methods themselves with learning. We conclude with future directions for formal methods in reinforcement learning, uncertainty, privacy, explainability of formal methods, and regulation and certification.

Georgios Bakirtzis, Michail Savvas, Ufuk Topcu

Compositional knowledge representations in reinforcement learning (RL) facilitate modular, interpretable, and safe task specifications. However, generating compositional models requires the characterization of minimal assumptions for the robustness of the compositionality feature, especially in the case of functional decompositions. Using a categorical point of view, we develop a knowledge representation framework for a compositional theory of RL. Our approach relies on the theoretical study of the category MDP, whose objects are Markov decision processes (MDPs) acting as models of tasks. The categorical semantics models the compositionality of tasks through the application of pushout operations akin to combining puzzle pieces. As a practical application of these pushout operations, we introduce zig-zag diagrams that rely on the compositional guarantees engendered by the category MDP. We further prove that properties of the category MDP unify concepts, such as enforcing safety requirements and exploiting symmetries, generalizing previous abstraction theories for RL.

Georgios Bakirtzis, Michail Savvas, Ruihan Zhao et al.

In reinforcement learning, conducting task composition by forming cohesive, executable sequences from multiple tasks remains challenging. However, the ability to (de)compose tasks is a linchpin in developing robotic systems capable of learning complex behaviors. Yet, compositional reinforcement learning is beset with difficulties, including the high dimensionality of the problem space, scarcity of rewards, and absence of system robustness after task composition. To surmount these challenges, we view task composition through the prism of category theory -- a mathematical discipline exploring structures and their compositional relationships. The categorical properties of Markov decision processes untangle complex tasks into manageable sub-tasks, allowing for strategical reduction of dimensionality, facilitating more tractable reward structures, and bolstering system robustness. Experimental results support the categorical theory of reinforcement learning by enabling skill reduction, reuse, and recycling when learning complex robotic arm tasks.

Georgios Bakirtzis, Andrea Aler Tubella, Andreas Theodorou et al.

Sociotechnical requirements shape the governance of artificially intelligent (AI) systems. In an era where embodied AI technologies are rapidly reshaping various facets of contemporary society, their inherent dynamic adaptability presents a unique blend of opportunities and challenges. Traditional regulatory mechanisms, often designed for static -- or slower-paced -- technologies, find themselves at a crossroads when faced with the fluid and evolving nature of AI systems. Moreover, typical problems in AI, for example, the frequent opacity and unpredictability of the behaviour of the systems, add additional sociotechnical challenges. To address these interconnected issues, we introduce the concept of dynamic certification, an adaptive regulatory framework specifically crafted to keep pace with the continuous evolution of AI systems. The complexity of these challenges requires common progress in multiple domains: technical, socio-governmental, and regulatory. Our proposed transdisciplinary approach is designed to ensure the safe, ethical, and practical deployment of AI systems, aligning them bidirectionally with the real-world contexts in which they operate. By doing so, we aim to bridge the gap between rapid technological advancement and effective regulatory oversight, ensuring that AI systems not only achieve their intended goals but also adhere to ethical standards and societal values.

Dong Ho Lee, Jingqi Li, Lasse Peters et al.

Games of ordered preference (GOOPs) model multi-player equilibrium problems in which each player maintains a distinct hierarchy of strictly prioritized objectives. Existing approaches solve GOOPs by deriving and enforcing the necessary optimality conditions that characterize lexicographically constrained Nash equilibria through a single-level reformulation. However, the number of primal and dual variables in the resulting KKT system grows exponentially with the number of preference levels, leading to severe scalability challenges. We derive a compact reformulation of these necessary conditions that preserves the essential primal stationarity structure across hierarchy levels, yielding a "reduced" KKT system whose size grows polynomially with both the number of players and the number of preference levels. The reduced system constitutes a relaxation of the complete KKT system, yet it remains a valid necessary condition for local GOOP equilibria. For GOOPs with quadratic objectives and linear constraints, we prove that the primal solution sets of the reduced and complete KKT systems coincide. More generally, for GOOPs with arbitrary (but smooth) nonlinear objectives and constraints, the reduced KKT conditions recover all local GOOP equilibria but may admit spurious non-equilibrium solutions. We introduce a second-order sufficient condition to certify when a candidate point corresponds to a local GOOP equilibrium. We also develop a primal-dual interior-point method for computing a local GOOP equilibrium with local quadratic convergence. The resulting framework enables scalable and efficient computation of GOOP equilibria beyond the tractable range of existing exponentially complex formulations.

Pau de las Heras Molins, Beyazit Yalcinkaya, Lasse Peters et al.

Multi-objective reinforcement learning (MORL) allows a user to express preference over outcomes in terms of the relative importance of the objectives, but standard metrics cannot capture whether changes in preference reliably change the agent's behavior in the intended way, a property termed controllability. As a result, preference-conditioned agents can score well on standard MORL metrics while being insensitive to the preference input. If the ability to control agents cannot be reliably assessed, the symbolic interface that MORL provides between user intent and agent behavior is broken. Mainstream MORL metrics alone fail to measure the controllability of preference-conditioned agents, motivating a complementary metric specifically designed to that end. We hope the results spur discussion in the community on existing evaluation protocols to consolidate advances in preference adaptation in MORL to larger and more complex problems.

Georgios Bakirtzis, Tim Sherburne, Stephen Adams et al.

System complexity has become ubiquitous in the design, assessment, and implementation of practical and useful cyber-physical systems. This increased complexity is impacting the management of models necessary for designing cyber-physical systems that are able to take into account a number of ``-ilities'', such that they are safe and secure and ultimately resilient to disruption of service. We propose an ontological metamodel for system design that augments an already existing industry metamodel to capture the relationships between various model elements and safety, security, and resilient considerations. Employing this metamodel leads to more cohesive and structured modeling efforts with an overall increase in scalability, usability, and unification of already existing models. In turn, this leads to a mission-oriented perspective in designing security defenses and resilience mechanisms to combat undesirable behaviors. We illustrate this metamodel in an open-source GraphQL implementation, which can interface with a number of modeling languages. We support our proposed metamodel with a detailed demonstration using an oil and gas pipeline model.

Georgios Bakirtzis

This dissertation builds a compositional cyber-physical systems theory to develop concrete semantics relating the above diverse views necessary for safety and security assurance. In this sense, composition can take two forms. The first is composing larger models from smaller ones within each individual formalism of requirements, behaviors, and architectures which can be thought of as horizontal composition -- a problem which is largely solved. The second and main contribution of this theory is vertical composition, meaning relating or otherwise providing verified composition across requirement, behavioral, and architecture models and their associated algebras. In this dissertation, we show that one possible solution to vertical composition is to use tools from category theory. Category theory is a natural candidate for making both horizontal and vertical composition formally explicit because it can relate, compare, and/or unify different algebras.

Georgios Bakirtzis, Fabrizio Genovese, Cody H. Fleming

Our work focuses on modeling the security of systems from their component-level designs. Towards this goal, we develop a categorical formalism to model attacker actions. Equipping the categorical formalism with algebras produces two interesting results for security modeling. First, using the Yoneda lemma, we can model attacker reconnaissance missions. In this context, the Yoneda lemma shows us that if two system representations, one being complete and the other being the attacker's incomplete view, agree at every possible test, they behave the same. The implication is that attackers can still successfully exploit the system even with incomplete information. Second, we model the potential changes to the system via an exploit. An exploit either manipulates the interactions between system components, such as providing the wrong values to a sensor, or changes the components themselves, such as controlling a global positioning system (GPS). One additional benefit of using category theory is that mathematical operations can be represented as formal diagrams, helpful in applying this analysis in a model-based design setting. We illustrate this modeling framework using an unmanned aerial vehicle (UAV) cyber-physical system model. We demonstrate and model two types of attacks (1) a rewiring attack, which violates data integrity, and (2) a rewriting attack, which violates availability.

Cody Fleming, Carl Elks, Georgios Bakirtzis et al.

Cyber-physical systems (CPS) are often defended in the same manner as information technology (IT) systems -- by using perimeter security. Multiple factors make such defenses insufficient for CPS. Resiliency shows potential in overcoming these shortfalls. Techniques for achieving resilience exist; however, methods and theory for evaluating resilience in CPS are lacking. We argue that such methods and theory should assist stakeholders in deciding where and how to apply design patterns for resilience. Such a problem potentially involves tradeoffs between different objectives and criteria, and such decisions need to be driven by traceable, defensible, repeatable engineering evidence. Multi-criteria resiliency problems require a system-oriented approach that evaluates systems in the presence of threats as well as potential design solutions once vulnerabilities have been identified. We present a systems-oriented view of cyber-physical security, termed Mission Aware, that is based on a holistic understanding of mission goals, system dynamics, and risk.

Georgios Bakirtzis, Garrett L. Ward, Christopher J. Deloglos et al.

Systems modeling practice lacks security analysis tools that can interface with modeling languages to facilitate security by design. Security by design is a necessity in the age of safety critical cyber-physical systems, where security violations can cause hazards. Currently, the overlap between security and safety is narrow. But deploying cyber-physical systems means that today's adversaries can intentionally trigger accidents. By implementing security assessment tools for modeling languages we are better able to address threats earlier in the system's lifecycle and, therefore, assure their safe and secure behavior in their eventual deployment. We posit that cyber-physical systems security modeling is practiced insufficiently because it is still addressed similarly to information technology systems.

Georgios Bakirtzis, Brandon J. Simon, Aidan G. Collins et al.

Applying security as a lifecycle practice is becoming increasingly important to combat targeted attacks in safety-critical systems. Among others there are two significant challenges in this area: (1) the need for models that can characterize a realistic system in the absence of an implementation and (2) an automated way to associate attack vector information; that is, historical data, to such system models. We propose the cybersecurity body of knowledge (CYBOK), which takes in sufficiently characteristic models of systems and acts as a search engine for potential attack vectors. CYBOK is fundamentally an algorithmic approach to vulnerability exploration, which is a significant extension to the body of knowledge it builds upon. By using CYBOK, security analysts and system designers can work together to assess the overall security posture of systems early in their lifecycle, during major design decisions and before final product designs. Consequently, assisting in applying security earlier and throughout the systems lifecycle.

Smitha Gautham, Georgios Bakirtzis, Matthew T. Leccadito et al.

Cyber-physical systems (CPS) are composed of various embedded subsystems and require specialized software, firmware, and hardware to coordinate with the rest of the system. These multiple levels of integration expose attack surfaces which can be susceptible to attack vectors that require novel architectural methods to effectively secure against. We present a multilevel hierarchical monitor architecture cybersecurity approach applied to a flight control system. However, the principles present in this paper apply to any CPS. Additionally, the real-time nature of these monitors allow for adaptable security, meaning that they mitigate against possible classes of attacks online. This results in an appealing bolt-on solution that is independent of different system designs. Consequently, employing such monitors leads to strengthened system resiliency and dependability of safety-critical CPS.

Georgios Bakirtzis, Brandon J. Simon, Cody H. Fleming et al.

Today, there is a plethora of software security tools employing visualizations that enable the creation of useful and effective interactive security analyst dashboards. Such dashboards can assist the analyst to understand the data at hand and, consequently, to conceive more targeted preemption and mitigation security strategies. Despite the recent advances, model-based security analysis is lacking tools that employ effective dashboards---to manage potential attack vectors, system components, and requirements. This problem is further exacerbated because model-based security analysis produces significantly larger result spaces than security analysis applied to realized systems---where platform specific information, software versions, and system element dependencies are known. Therefore, there is a need to manage the analysis complexity in model-based security through better visualization techniques. Towards that goal, we propose an interactive security analysis dashboard that provides different views largely centered around the system, its requirements, and its associated attack vector space. This tool makes it possible to start analysis earlier in the system lifecycle. We apply this tool in a significant area of engineering design---the design of cyber-physical systems---where security violations can lead to safety hazards.

Georgios Bakirtzis, Bryan T. Carter, Cody H. Fleming et al.

Perimeter cybersecurity, while essential, has proven insufficient against sophisticated, coordinated, and cyber-physical attacks. In contrast, mission-centric cybersecurity emphasizes finding evidence of attack impact on mission success, allowing for targeted resource allocation to mitigate vulnerabilities and protect critical assets. Mission Aware is a systems-theoretic cybersecurity analysis that identifies components which, if compromised, destabilize the overall mission. It generates evidence by finding potential attack vectors relevant to mission-linked elements and traces this evidence to mission requirements, prioritizing high-impact vulnerabilities relative to mission objectives. Mission Aware is an informational tool for system resilience by unifying cybersecurity analysis with core systems engineering goals.

Bryan Carter, Georgios Bakirtzis, Carl Elks et al.

The security of cyber-physical systems is first and foremost a safety problem, yet it is typically handled as a traditional security problem, which means that solutions are based on defending against threats and are often implemented too late. This approach neglects to take into consideration the context in which the system is intended to operate, thus system safety may be compromised. This paper presents a systems-theoretic analysis approach that combines stakeholder perspectives with a modified version of Systems-Theoretic Accident Model and Process (STAMP) that allows decision-makers to strategically enhance the safety, resilience, and security of a cyber-physical system against potential threats. This methodology allows the capture of vital mission-specific information in a model, which then allows analysts to identify and mitigate vulnerabilities in the locations most critical to mission success. We present an overview of the general approach followed by a real example using an unmanned aerial vehicle conducting a reconnaissance mission.

Georgios Bakirtzis, Bryan T. Carter, Carl R. Elks et al.

Evaluating the security of cyber-physical systems throughout their life cycle is necessary to assure that they can be deployed and operated in safety-critical applications, such as infrastructure, military, and transportation. Most safety and security decisions that can have major effects on mitigation strategy options after deployment are made early in the system's life cycle. To allow for a vulnerability analysis before deployment, a sufficient well-formed model has to be constructed. To construct such a model we produce a taxonomy of attributes; that is, a generalized schema for system attributes. This schema captures the necessary specificity that characterizes a possible real system and can also map to the attack vector space associated with the model's attributes. In this way, we can match possible attack vectors and provide architectural mitigation at the design phase. We present a model of a flight control system encoded in the Systems Modeling Language, commonly known as SysML, but also show agnosticism with respect to the modeling language or tool used.