Janos Sztipanovits

Harsh Vardhan, David Hyde, Umesh Timalsina et al.

Physics simulations like computational fluid dynamics (CFD) are a computational bottleneck in computer-aided design (CAD) optimization processes. To overcome this bottleneck, one requires either an optimization framework that is highly sample-efficient, or a fast data-driven proxy (surrogate model) for long-running simulations. Both approaches have benefits and limitations. Bayesian optimization is often used for sample efficiency, but it solves one specific problem and struggles with transferability; alternatively, surrogate models can offer fast and often more generalizable solutions for CFD problems, but gathering data for and training such models can be computationally demanding. In this work, we leverage recent advances in optimization and artificial intelligence (AI) to explore both of these potential approaches, in the context of designing an optimal unmanned underwater vehicle (UUV) hull. Our study finds that the Bayesian Optimization-Lower Condition Bound (BO-LCB) algorithm is the most sample-efficient optimization framework and has the best convergence behavior of those considered. Subsequently, we show that our DNN-based surrogate model predicts drag force on test data in tight agreement with CFD simulations, with a mean absolute percentage error (MAPE) of 1.85%. Combining these results, we demonstrate a two-orders-of-magnitude speedup (with comparable accuracy) for the design optimization process when the surrogate model is used. To our knowledge, this is the first study applying Bayesian optimization and DNN-based surrogate modeling to the problem of UUV design optimization, and we share our developments as open-source software.

Harsh Vardhan, Janos Sztipanovits

Designing an inexpensive approximate surrogate model that captures the salient features of an expensive high-fidelity behavior is a prevalent approach in design optimization. In recent times, Deep Learning (DL) models are being used as a promising surrogate computational model for engineering problems. However, the main challenge in creating a DL-based surrogate is to simulate/label a large number of design points, which is time-consuming for computationally costly and/or high-dimensional engineering problems. In the present work, we propose a novel sampling technique by combining the active learning (AL) method with DL. We call this method $ε$-weighted hybrid query strategy ($ε$-HQS) , which focuses on the evaluation of the surrogate at each learning iteration and provides an estimate of the failure probability of the surrogate in the Design Space. By reusing already collected training and test data, the learned failure probability guides the next iteration's sampling process to the region of the high probability of failure. During the empirical evaluation, better accuracy of the surrogate was observed in comparison to other methods of sample selection. We empirically evaluated this method in two different engineering design domains, finite element based static stress analysis of submarine pressure vessel(computationally costly process) and second submarine propeller design( high dimensional problem). https://github.com/vardhah/epsilon_weighted_Hybrid_Query_Strategy

Harsh Vardhan, Janos Sztipanovits

Machine learning models have prevalent applications in many real-world problems, which increases the importance of correctness in the behaviour of these trained models. Finding a good test case that can reveal the potential failure in these trained systems can help to retrain these models to increase their correctness. For a well-trained model, the occurrence of a failure is rare. Consequently, searching these rare scenarios by evaluating each sample in input search space or randomized search would be costly and sometimes intractable due to large search space, limited computational resources, and available time. In this paper, we tried to address this challenge of finding these failure scenarios faster than traditional randomized search. The central idea of our approach is to separate the input data space in region of high failure probability and region of low/minimal failure probability based on the observation made by training data, data drawn from real-world statistics, and knowledge from a domain expert. Using these information, we can design a generative model from which we can generate scenarios that have a high likelihood to reveal the potential failure. We evaluated this approach on two different experimental scenarios and able to speed up the discovery of such failures a thousand-fold faster than the traditional randomized search.

Harsh Vardhan, Peter Volgyesi, Will Hedgecock et al.

Automatic underwater vehicle hull Design optimization is a complex engineering process for generating a UUV hull with optimized properties on a given requirement. First, it involves the integration of involved computationally complex engineering simulation tools. Second, it needs integration of a sample efficient optimization framework with the integrated toolchain. To this end, we integrated the CAD tool called FreeCAD with CFD tool openFoam for automatic design evaluation. For optimization, we chose Bayesian optimization (BO), which is a well-known technique developed for optimizing time-consuming expensive engineering simulations and has proven to be very sample efficient in a variety of problems, including hyper-parameter tuning and experimental design. During the optimization process, we can handle infeasible design as constraints integrated into the optimization process. By integrating domain-specific toolchain with AI-based optimization, we executed the automatic design optimization of underwater vehicle hull design. For empirical evaluation, we took two different use cases of real-world underwater vehicle design to validate the execution of our tool.

Sarthak Kapoor, Harsh Vardhan, Umesh Timalsina et al.

High fidelity design evaluation processes such as Computational Fluid Dynamics and Finite Element Analysis are often replaced with data driven surrogates to reduce computational cost in engineering design optimization. However, building accurate surrogate models still requires a large number of expensive simulations. To address this challenge, we introduce epsilon HQS, a scalable active learning strategy that leverages a student teacher framework to train deep neural networks efficiently. Unlike Bayesian AL methods, which are computationally demanding with DNNs, epsilon HQS selectively queries informative samples to reduce labeling cost. Applied to CFD, FEA, and propeller design tasks, our method achieves higher accuracy under fixed labeling cost budgets.

Harsh Vardhan, Janos Sztipanovits

In Autonomous Underwater Vehicles (AUVs) design, hull resistance is an important factor in determining the power requirements and range of vehicle and consequently affect battery size, weight, and volume requirement of the design. In this paper, we leverage on AI-based optimization algorithm along with Computational Fluid Dynamics (CFD) simulation to study the optimal hull design that minimizing the resistance. By running the CFD-based optimization at different operating velocities and turbulence intensity, we want to study/search the possibility of a universal design that will provide least resistance/near-optimal design across all operating conditions (operating velocity) and environmental conditions (turbulence intensity). Early result demonstrated that the optimal design found at low velocity and low turbulence condition performs very poor at high velocity and high turbulence conditions. However, a design that is optimal at high velocity and high turbulence conditions performs near-optimal across many considered velocity and turbulence conditions.

Harsh Vardhan, Janos Sztipanovits

Most machine learning-based regressors extract information from data collected via past observations of limited length to make predictions in the future. Consequently, when input to these trained models is data with significantly different statistical properties from data used for training, there is no guarantee of accurate prediction. Consequently, using these models on out-of-distribution input data may result in a completely different predicted outcome from the desired one, which is not only erroneous but can also be hazardous in some cases. Successful deployment of these machine learning models in any system requires a detection system, which should be able to distinguish between out-of-distribution and in-distribution data (i.e. similar to training data). In this paper, we introduce a novel approach for this detection process using a Reduced Robust Random Cut Forest (RRRCF) data structure, which can be used on both small and large data sets. Similar to the Robust Random Cut Forest (RRCF), RRRCF is a structured, but a reduced representation of the training data sub-space in form of cut trees. Empirical results of this method on both low and high-dimensional data showed that inference about data being in/out of training distribution can be made efficiently and the model is easy to train with no difficult hyper-parameter tuning. The paper discusses two different use-cases for testing and validating results.

Harsh Vardhan, Peter Volgyesi, Janos Sztipanovits

In computer-aided engineering design, the goal of a designer is to find an optimal design on a given requirement using the numerical simulator in loop with an optimization method. In this design optimization process, a good design optimization process is one that can reduce the time from inception to design. In this work, we take a class of design problem, that is computationally cheap to evaluate but has high dimensional design space. In such cases, traditional surrogate-based optimization does not offer any benefits. In this work, we propose an alternative way to use ML model to surrogate the design process that formulates the search problem as an inverse problem and can save time by finding the optimal design or at least a good initial seed design for optimization. By using this trained surrogate model with the traditional optimization method, we can get the best of both worlds. We call this as Surrogate Assisted Optimization (SAO)- a hybrid approach by mixing ML surrogate with the traditional optimization method. Empirical evaluations of propeller design problems show that a better efficient design can be found in fewer evaluations using SAO.

Harsh Vardhan, Janos Sztipanovits

During the design process of an autonomous underwater vehicle (AUV), the pressure vessel has a critical role. The pressure vessel contains dry electronics, power sources, and other sensors that can not be flooded. A traditional design approach for a pressure vessel design involves running multiple Finite Element Analysis (FEA) based simulations and optimizing the design to find the best suitable design which meets the requirement. Running these FEAs are computationally very costly for any optimization process and it becomes difficult to run even hundreds of evaluation. In such a case, a better approach is the surrogate design with the goal of replacing FEA-based prediction with some learning-based regressor. Once the surrogate is trained for a class of problem, then the learned response surface can be used to analyze the stress effect without running the FEA for that class of problem. The challenge of creating a surrogate for a class of problems is data generation. Since the process is computationally costly, it is not possible to densely sample the design space and the learning response surface on sparse data set becomes difficult. During experimentation, we observed that a Deep Learning-based surrogate outperforms other regression models on such sparse data. In the present work, we are utilizing the Deep Learning-based model to replace the costly finite element analysis-based simulation process. By creating the surrogate we speed up the prediction on the other design much faster than direct Finite element Analysis. We also compared our DL-based surrogate with other classical Machine Learning (ML) based regression models( random forest and Gradient Boost regressor). We observed on the sparser data, the DL-based surrogate performs much better than other regression models.

Harsh Vardhan, Umesh Timalsina, Michael Sandborn et al.

In this work, we introduce an open-source integrated CAD-CFD tool, Anvil, which combines FreeCAD for CAD modeling and OpenFOAM for CFD analysis, along with an AI-based optimization method (Bayesian optimization) and other sampling algorithms. Anvil serves as a scientific machine learning tool for shape optimization in three modes: data generation, CFD evaluation, and shape optimization. In data generation mode, it automatically runs CFD evaluations and generates data for training a surrogate model. In optimization mode, it searches for the optimal design under given requirements and optimization metrics. In CFD mode, a single CAD file can be evaluated with a single OpenFOAM run. To use Anvil, experimenters provide a JSON configuration file and a parametric CAD seed design. Anvil can be used to study solid-fluid dynamics for any subsonic flow conditions and has been demonstrated in various simulation and optimization use cases. The open-source code for the tool, installation process, artifacts (such as CAD seed designs and example STL models), experimentation results, and detailed documentation can be found at \url{https://github.com/symbench/Anvil}.

Anastasia Mavridou, Joseph Sifakis, Janos Sztipanovits

The Behavior-Interaction-Priority (BIP) framework, rooted in rigorous semantics, allows the construction of systems that are correct-by-design. BIP has been effectively used for the construction and analysis of large systems such as robot controllers and satellite on-board software. Nevertheless, the specification of BIP models is done in a purely textual manner without any code editor support. To facilitate the specification of BIP models, we present DesignBIP, a web-based, collaborative, version-controlled design studio. To promote model scaling and reusability of BIP models, we use a graphical language for modeling parameterized BIP models with rigorous semantics. We present the various services provided by the design studio, including model editors, code editors, consistency checking mechanisms, code generators, and integration with the JavaBIP tool-set.

Anastasia Mavridou, Joseph Sifakis, Janos Sztipanovits

The Behavior-Interaction-Priority (BIP) framework, rooted in rigorous semantics, allows the construction of systems that are correct-by-design. BIP has been effectively used for the construction and analysis of large systems such as robot controllers and satellite on-board software. Nevertheless, the specification of BIP models is done in a purely textual manner without any code editor support. To facilitate the specification of BIP models, we present DesignBIP, a web-based, collaborative, version-controlled design studio. To promote model scaling and reusability of BIP models, we use a graphical language for modeling parameterized BIP models with rigorous semantics. We present the various services provided by the design studio, including model editors, code editors, consistency checking mechanisms, code generators, and integration with the JavaBIP tool-set.

Alexander Kott, Benjamin Blakely, Diane Henshel et al.

This report summarizes the discussions and findings of the 2017 North Atlantic Treaty Organization (NATO) Workshop, IST-153, on Cyber Resilience, held in Munich, Germany, on 23-25 October 2017, at the University of Bundeswehr. Despite continual progress in managing risks in the cyber domain, anticipation and prevention of all possible attacks and malfunctions are not feasible for the current or future systems comprising the cyber infrastructure. Therefore, interest in cyber resilience (as opposed to merely risk-based approaches) is increasing rapidly, in literature and in practice. Unlike concepts of risk or robustness - which are often and incorrectly conflated with resilience - resiliency refers to the system's ability to recover or regenerate its performance to a sufficient level after an unexpected impact produces a degradation of its performance. The exact relation among resilience, risk, and robustness has not been well articulated technically. The presentations and discussions at the workshop yielded this report. It focuses on the following topics that the participants of the workshop saw as particularly important: fundamental properties of cyber resilience; approaches to measuring and modeling cyber resilience; mission modeling for cyber resilience; systems engineering for cyber resilience, and dynamic defense as a path toward cyber resilience.

Nafees Qamar, Yilong Yang, Andras Nadas et al.

This paper takes on the problem of automatically identifying clinically-relevant patterns in medical datasets without compromising patient privacy. To achieve this goal, we treat datasets as a black box for both internal and external users of data that lets us handle clinical data queries directly and far more efficiently. The novelty of the approach lies in avoiding the data de-identification process often used as a means of preserving patient privacy. The implemented toolkit combines software engineering technologies such as Java EE and RESTful web services, to allow exchanging medical data in an unidentifiable XML format as well as restricting users to the need-to-know principle. Our technique also inhibits retrospective processing of data, such as attacks by an adversary on a medical dataset using advanced computational methods to reveal Protected Health Information (PHI). The approach is validated on an endoscopic reporting application based on openEHR and MST standards. From the usability perspective, the approach can be used to query datasets by clinical researchers, governmental or non-governmental organizations in monitoring health care services to improve quality of care.