Han-Lim Choi

Su-Jin Lee, Soon-Seo Park, Han-Lim Choi

This paper presents a potential game-based method for non-myopic planning of mobile sensor networks in the context of target tracking. The planning objective is to select the sequence of sensing points over more than one future time steps to maximize information about the target states. This multi-step lookahead scheme is studied to overcome getting trapped at local information maximum when there are gaps in sensing coverage due to constraints on the sensor platform mobility or limitations in sensing capabilities. However, the long-term planning becomes computationally intractable as the length of planing horizon increases. This work develops a gametheoretic approach to address the computational challenges. The main contributions of this paper are twofold: (a) to formulate a non-myopic planning problem for tracking multiple targets into a potential game, the size of which linearly increases as the number of planning steps (b) to design a learning algorithm exploiting the joint strategy fictitious play and dynamic programming, which overcomes the gaps in sensing coverage. Numerical examples of multi-target tracking demonstrate that the proposed method gives better estimation performance than myopic planning and is computationally tractable.

Lebsework Negash, Sang-Hyeon Kim, Han-Lim Choi

In this paper, cyber attack detection and isolation is studied on a network of UAVs in a formation flying setup. As the UAVs communicate to reach consensus on their states while making the formation, the communication network among the UAVs makes them vulnerable to a potential attack from malicious adversaries. Two types of attacks pertinent to a network of UAVs have been considered: a node attack on the UAVs and a deception attack on the communication between the UAVs. UAVs formation control presented using a consensus algorithm to reach a pre-specified formation. A node and a communication path deception cyber attacks on the UAV's network are considered with their respective models in the formation setup. For these cyber attacks detection, a bank of Unknown Input Observer (UIO) based distributed fault detection scheme proposed to detect and identify the compromised UAV in the formation. A rule based on the residuals generated using the bank of UIOs are used to detect attacks and identify the compromised UAV in the formation. Further, an algorithm developed to remove the faulty UAV from the network once an attack detected and the compromised UAV isolated while maintaining the formation flight with a missing UAV node.

Thaweerath Phisannupawong, Joshua Julian Damanik, Han-Lim Choi

Air traffic trajectory recognition has gained significant interest within the air traffic management community, particularly for fundamental tasks such as classification and clustering. This paper introduces Aircraft Trajectory Segmentation-based Contrastive Coding (ATSCC), a novel self-supervised time series representation learning framework designed to capture semantic information in air traffic trajectory data. The framework leverages the segmentable characteristic of trajectories and ensures consistency within the self-assigned segments. Intensive experiments were conducted on datasets from three different airports, totaling four datasets, comparing the learned representation's performance of downstream classification and clustering with other state-of-the-art representation learning techniques. The results show that ATSCC outperforms these methods by aligning with the labels defined by aeronautical procedures. ATSCC is adaptable to various airport configurations and scalable to incomplete trajectories. This research has expanded upon existing capabilities, achieving these improvements independently without predefined inputs such as airport configurations, maneuvering procedures, or labeled data.

Min Kyu Shin, Su-Jeong Park, Seung-Keol Ryu et al.

This paper presents a novel learning approach for Dubins Traveling Salesman Problems(DTSP) with Neighborhood (DTSPN) to quickly produce a tour of a non-holonomic vehicle passing through neighborhoods of given task points. The method involves two learning phases: initially, a model-free reinforcement learning approach leverages privileged information to distill knowledge from expert trajectories generated by the LinKernighan heuristic (LKH) algorithm. Subsequently, a supervised learning phase trains an adaptation network to solve problems independently of privileged information. Before the first learning phase, a parameter initialization technique using the demonstration data was also devised to enhance training efficiency. The proposed learning method produces a solution about 50 times faster than LKH and substantially outperforms other imitation learning and RL with demonstration schemes, most of which fail to sense all the task points.

Thaweerath Phisannupawong, Joshua Julian Damanik, Han-Lim Choi

Flight delay prediction has become a key focus in air traffic management, as delays highlight inefficiencies that impact overall network performance. This paper presents a lightweight large language model-based multimodal flight delay prediction, formulated from the perspective of air traffic controllers monitoring aircraft delay after entering the terminal area. The approach integrates trajectory representations with textual aeronautical information, including flight information, weather reports, and aerodrome notices, by adapting trajectory data into the language modality to capture airspace conditions. The experiments show that the model consistently achieves sub-minute prediction error by effectively leveraging contextual information related to the sources of delay, fulfilling the operational standard for minute-level precision. The framework demonstrates that linguistic understanding, when combined with cross-modality adaptation of trajectory data, enhances delay prediction. Moreover, the approach shows practicality and potential scalability for real-world operations, supporting real-time updates that refine predictions upon receiving new operational information.

Jung-Su Ha, Young-Jin Park, Hyeok-Joo Chae et al.

We present a hierarchical planning and control framework that enables an agent to perform various tasks and adapt to a new task flexibly. Rather than learning an individual policy for each particular task, the proposed framework, DISH, distills a hierarchical policy from a set of tasks by representation and reinforcement learning. The framework is based on the idea of latent variable models that represent high-dimensional observations using low-dimensional latent variables. The resulting policy consists of two levels of hierarchy: (i) a planning module that reasons a sequence of latent intentions that would lead to an optimistic future and (ii) a feedback control policy, shared across the tasks, that executes the inferred intention. Because the planning is performed in low-dimensional latent space, the learned policy can immediately be used to solve or adapt to new tasks without additional training. We demonstrate the proposed framework can learn compact representations (3- and 1-dimensional latent states and commands for a humanoid with 197- and 36-dimensional state features and actions) while solving a small number of imitation tasks, and the resulting policy is directly applicable to other types of tasks, i.e., navigation in cluttered environments. Video: https://youtu.be/HQsQysUWOhg

Soon-Seo Park, Young-Jin Park, Youngjae Min et al.

This paper proposes an online learning method of Gaussian process state-space model (GP-SSM). GP-SSM is a probabilistic representation learning scheme that represents unknown state transition and/or measurement models as Gaussian processes (GPs). While the majority of prior literature on learning of GP-SSM are focused on processing a given set of time series data, data may arrive and accumulate sequentially over time in most dynamical systems. Storing all such sequential data and updating the model over entire data incur large amount of computational resources in space and time. To overcome this difficulty, we propose a practical method, termed \textit{onlineGPSSM}, that incorporates stochastic variational inference (VI) and online VI with novel formulation. The proposed method mitigates the computational complexity without catastrophic forgetting and also support adaptation to changes in a system and/or a real environments. Furthermore, we present application of onlineGPSSM into the reinforcement learning (RL) of partially observable dynamical systems by integrating onlineGPSSM with Bayesian filtering and trajectory optimization algorithms. Numerical examples are presented to demonstrate applicability of the proposed method.

Young-Jin Park, Han-Lim Choi

The goal of system identification is to learn about underlying physics dynamics behind the time-series data. To model the probabilistic and nonparametric dynamics model, Gaussian process (GP) have been widely used; GP can estimate the uncertainty of prediction and avoid over-fitting. Traditional GPSSMs, however, are based on Gaussian transition model, thus often have difficulty in describing a more complex transition model, e.g. aircraft motions. To resolve the challenge, this paper proposes a framework using multiple GP transition models which is capable of describing multi-modal dynamics. Furthermore, we extend the model to the information-theoretic framework, the so-called InfoSSM, by introducing a mutual information regularizer helping the model to learn interpretable and distinguishable multiple dynamics models. Two illustrative numerical experiments in simple Dubins vehicle and high-fidelity flight simulator are presented to demonstrate the performance and interpretability of the proposed model. Finally, this paper introduces a framework using InfoSSM with Bayesian filtering for air traffic control tracking.

Doo-Hyun Cho, Dae-Sung Jang, Han-Lim Choi

This paper describes a formulation and develops a novel procedure for a fleet of unmanned aerial vehicles (UAVs) from the perspective of remotely executable tasks. In a complex mission environment, the characteristics of vehicles can be different in terms of sensing capability, range, direction, or the motion constraints. The purpose of this paper is to find a set of paths that minimizes the sum of costs while every task region is visited exactly once under certain reasonable assumptions. The heterogeneous multi-UAV path planning problem is formulated as a generalized, heterogeneous, multiple depot traveling salesmen problem (GHMDATSP), which is a variant of the traveling salesman problem. The proposed transformation procedure changes an instance of the GHMDATSP into a format of an Asymmetric, Traveling Salesman Problem (ATSP) to obtain tours for which the total cost of a fleet of vehicles is minimized. The instance of the ATSP is solved using the Lin-Kernighan-Helsgaun heuristic, and the result is inversely transformed to the GHMDATSP-formatted instance to obtain a set of tours. An additional local optimization based path refinement process helps obtain a high-quality solution. Numerical experiments investigate and confirm for the validity and applicability of the proposed procedure.

Soon-Seo Park, Youngjae Min, Jung-Su Ha et al.

This paper investigates non-myopic path planning of mobile sensors for multi-target tracking. Such problem has posed a high computational complexity issue and/or the necessity of high-level decision making. Existing works tackle these issues by heuristically assigning targets to each sensing agent and solving the split problem for each agent. However, such heuristic methods reduce the target estimation performance in the absence of considering the changes of target state estimation along time. In this work, we detour the task-assignment problem by reformulating the general non-myopic planning problem to a distributed optimization problem with respect to targets. By combining alternating direction method of multipliers (ADMM) and local trajectory optimization method, we solve the problem and induce consensus (i.e., high-level decisions) automatically among the targets. In addition, we propose a modified receding-horizon control (RHC) scheme and edge-cutting method for efficient real-time operation. The proposed algorithm is validated through simulations in various scenarios.

Jung-Su Ha, Young-Jin Park, Hyeok-Joo Chae et al.

We present a representation learning algorithm that learns a low-dimensional latent dynamical system from high-dimensional \textit{sequential} raw data, e.g., video. The framework builds upon recent advances in amortized inference methods that use both an inference network and a refinement procedure to output samples from a variational distribution given an observation sequence, and takes advantage of the duality between control and inference to approximately solve the intractable inference problem using the path integral control approach. The learned dynamical model can be used to predict and plan the future states; we also present the efficient planning method that exploits the learned low-dimensional latent dynamics. Numerical experiments show that the proposed path-integral control based variational inference method leads to tighter lower bounds in statistical model learning of sequential data. The supplementary video: https://youtu.be/xCp35crUoLQ

Amritha Prasad, Han-Lim Choi, Shreyas Sundaram

We consider a scenario consisting of a set of heterogeneous mobile agents located at a depot, and a set of tasks dispersed over a geographic area. The agents are partitioned into different types. The tasks are partitioned into specialized tasks that can only be done by agents of a certain type, and generic tasks that can be done by any agent. The distances between each pair of tasks are specified, and satisfy the triangle inequality. Given this scenario, we address the problem of allocating these tasks among the available agents (subject to type compatibility constraints) while minimizing the maximum cost to tour the allocation by any agent and return to the depot. This problem is NP-hard, and we give a three phase algorithm to solve this problem that provides 5-factor approximation, regardless of the total number of agents and the number of agents of each type. We also show that in the special case where there is only one agent of each type, the algorithm has an approximation factor of 4.

Jung-Su Ha, Hyeok-Joo Chae, Han-Lim Choi

This work presents an efficient framework to generate a motion plan of a robot with high degrees of freedom (e.g., a humanoid robot). High-dimensionality of the robot configuration space often leads to difficulties in utilizing the widely-used motion planning algorithms, since the volume of the decision space increases exponentially with the number of dimensions. To handle complications arising from the large decision space, and to solve a corresponding motion planning problem efficiently, two key concepts are adopted in this work: First, the Gaussian process latent variable model (GP-LVM) is utilized for low-dimensional representation of the original configuration space. Second, an approximate inference algorithm is used, exploiting through the duality between control and estimation, to explore the decision space and to compute a high-quality motion trajectory of the robot. Utilizing the GP-LVM and the duality between control and estimation, we construct a fully probabilistic generative model with which a high-dimensional motion planning problem is transformed into a tractable inference problem. Finally, we compute the motion trajectory via an approximate inference algorithm based on a variant of the particle filter. The resulting motions can be viewed in the supplemental video. ( https://youtu.be/kngEaOR4Esc )

Dae-Sung Jang, Hyeok-Joo Chae, Han-Lim Choi

This paper addresses path planning of an unmanned aerial vehicle (UAV) with remote sensing capabilities (or wireless communication capabilities). The goal of the path planning is to find a minimum-flight-time closed tour of the UAV visiting all executable areas of given remote sensing and communication tasks; in order to incorporate the nonlinear vehicle dynamics, this problem is regarded as a dynamically-constrained traveling salesman problem with neighborhoods. To obtain a close-to-optimal solution for the path planning in a tractable manner, a sampling-based roadmap algorithm that embeds an optimal control-based path generation process is proposed. The algorithm improves the computational efficiency by reducing numerical computations required for optimizing inefficient local paths, and by extracting additional information from a roadmap of a fixed number of samples. Comparative numerical simulations validate the efficiency of the presented algorithm in reducing computation time and improving the solution quality compared to previous roadmap-based planning methods.

Jung-Su Ha, Han-Lim Choi

This work presents a multiscale framework to solve an inverse reinforcement learning (IRL) problem for continuous-time/state stochastic systems. We take advantage of a diffusion wavelet representation of the associated Markov chain to abstract the state space. This not only allows for effectively handling the large (and geometrically complex) decision space but also provides more interpretable representations of the demonstrated state trajectories and also of the resulting policy of IRL. In the proposed framework, the problem is divided into the global and local IRL, where the global approximation of the optimal value functions are obtained using coarse features and the local details are quantified using fine local features. An illustrative numerical example on robot path control in a complex environment is presented to verify the proposed method.

Jung-Su Ha, Han-Lim Choi

This work presents a multiscale framework to solve a class of stochastic optimal control problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision space and complex environmental geometry, two key concepts are adopted: (a) a diffusion wavelet representation of the Markov chain for hierarchical abstraction of the state space; and (b) a desirability function-based representation of the Markov decision process (MDP) to efficiently calculate the optimal policy. In the proposed framework, a global plan that compressively takes into account the long time/length-scale state transition is first obtained by approximately solving an MDP whose desirability function is represented by coarse scale bases in the hierarchical abstraction. Then, a detailed local plan is computed by solving an MDP that considers wavelet bases associated with a focused region of the state space, guided by the global plan. The resulting multiscale plan is utilized to finally compute a continuous-time optimal control policy within a receding horizon implementation. Two numerical examples are presented to demonstrate the applicability and validity of the proposed approach.

Doo-Hyun Cho, Jung-Su Ha, Sujin Lee et al.

Informative path planning (IPP) is used to design paths for robotic sensor platforms to extract the best/maximum possible information about a quantity of interest while operating under a set of constraints, such as the dynamic feasibility of vehicles. The key challenges of IPP are the strong coupling in multiple layers of decisions: the selection of locations to visit, the allocation of sensor platforms to those locations; and the processing of the gathered information along the paths. This paper presents a systematic procedure for IPP and environmental mapping using multiple UAV sensor platforms. It (a) selects the best locations to observe, (b) calculates the cost and finds the best paths for each UAV, and (c) estimates the measurement value within a given region using the Gaussian process (GP) regression framework. An illustrative example of RF intensity field mapping is presented to demonstrate the validity and applicability of the proposed approach.

Jung-Su Ha, Soon-Seo Park, Han-Lim Choi

This paper addresses planning and control of robot motion under uncertainty that is formulated as a continuous-time, continuous-space stochastic optimal control problem, by developing a topology-guided path integral control method. The path integral control framework, which forms the backbone of the proposed method, re-writes the Hamilton-Jacobi-Bellman equation as a statistical inference problem; the resulting inference problem is solved by a sampling procedure that computes the distribution of controlled trajectories around the trajectory by the passive dynamics. For motion control of robots in a highly cluttered environment, however, this sampling can easily be trapped in a local minimum unless the sample size is very large, since the global optimality of local minima depends on the degree of uncertainty. Thus, a homology-embedded sampling-based planner that identifies many (potentially) local-minimum trajectories in different homology classes is developed to aid the sampling process. In combination with a receding-horizon fashion of the optimal control the proposed method produces a dynamically feasible and collision-free motion plans without being trapped in a local minimum. Numerical examples on a synthetic toy problem and on quadrotor control in a complex obstacle field demonstrate the validity of the proposed method.

Jung-Su Ha, Han-Lim Choi, Jeong hwan Jeon

This paper extends the RRT* algorithm, a recently developed but widely-used sampling-based optimal motion planner, in order to effectively handle nonlinear kinodynamic constraints. Nonlinearity in kinodynamic differential constraints often leads to difficulties in choosing appropriate distance metric and in computing optimized trajectory segments in tree construction. To tackle these two difficulties, this work adopts the affine quadratic regulator-based pseudo metric as the distance measure and utilizes iterative two-point boundary value problem solvers for computing the optimized segments. The proposed extension then preserves the inherent asymptotic optimality of the RRT* framework, while efficiently handling a variety of kinodynamic constraints. Three numerical case studies validate the applicability of the proposed method.

Jung-Su Ha, Han-Lim Choi

This work presents an efficient method to solve a class of continuous-time, continuous-space stochastic optimal control problems of robot motion in a cluttered environment. The method builds upon a path integral representation of the stochastic optimal control problem that allows computation of the optimal solution through sampling and estimation process. As this sampling process often leads to a local minimum especially when the state space is highly non-convex due to the obstacle field, we present an efficient method to alleviate this issue by devising a proposed topological motion planning algorithm. Combined with a receding-horizon scheme in execution of the optimal control solution, the proposed method can generate a dynamically feasible and collision-free trajectory while reducing concern about local optima. Illustrative numerical examples are presented to demonstrate the applicability and validity of the proposed approach.

Dae-Sung Jang, Han-Lim Choi

This paper presents heuristic algorithms for interleaved pulse scheduling problems on multi-target tracking in pulse Doppler phased array radars that can process multiple simultaneous received beams. The interleaved pulse scheduling problems for element and subarray level digital beamforming architectures are formulated as the same integer program and the asymptotic time complexities of the algorithms are analyzed.