SYAug 4, 2011
On the scalability and convergence of simultaneous parameter identification and synchronization of dynamical systemsBruno Nery, Rodrigo Ventura
The synchronization of dynamical systems is a method that allows two systems to have identical state trajectories, appart from an error converging to zero. This method consists in an appropriate unidirectional coupling from one system (drive) to the other (response). This requires that the response system shares the same dynamical model with the drive. For the cases where the drive is unknown, Chen proposed in 2002 a method to adapt the response system such that synchronization is achieved, provided that (1) the response dynamical model is linear with a vector of parameters, and (2) there is a parameter vector that makes both system dynamics identical. However, this method has two limitations: first, it does not scale well for complex parametric models (e.g., if the number of parameters is greater than the state dimension), and second, the model parameters are not guaranteed to converge, namely as the synchronization error approaches zero. This paper presents an adaptation law addressing these two limitations. Stability and convergence proofs, using Lyapunov's second method, support the proposed adaptation law. Finally, numerical simulations illustrate the advantages of the proposed method, namely showing cases where the Chen's method fail, while the proposed one does not.
37.5LGApr 2Code
Causal-Audit: A Framework for Risk Assessment of Assumption Violations in Time-Series Causal DiscoveryMarco Ruiz, Miguel Arana-Catania, David R. Ardila et al.
Time-series causal discovery methods rely on assumptions such as stationarity, regular sampling, and bounded temporal dependence. When these assumptions are violated, structure learning can produce confident but misleading causal graphs without warning. We introduce Causal-Audit, a framework that formalizes assumption validation as calibrated risk assessment. The framework computes effect-size diagnostics across five assumption families (stationarity, irregularity, persistence, nonlinearity, and confounding proxies), aggregates them into four calibrated risk scores with uncertainty intervals, and applies an abstention-aware decision policy that recommends methods (e.g., PCMCI+, VAR-based Granger causality) only when evidence supports reliable inference. The semi-automatic diagnostic stage can also be used independently for structured assumption auditing in individual studies. Evaluation on a synthetic atlas of 500 data-generating processes (DGPs) spanning 10 violation families demonstrates well-calibrated risk scores (AUROC > 0.95), a 62% false positive reduction among recommended datasets, and 78% abstention on severe-violation cases. On 21 external evaluations from TimeGraph (18 categories) and CausalTime (3 domains), recommend-or-abstain decisions are consistent with benchmark specifications in all cases. An open-source implementation of our framework is available.
CVSep 26, 2024
AI-Powered Augmented Reality for Satellite Assembly, Integration and TestAlvaro Patricio, Joao Valente, Atabak Dehban et al.
The integration of Artificial Intelligence (AI) and Augmented Reality (AR) is set to transform satellite Assembly, Integration, and Testing (AIT) processes by enhancing precision, minimizing human error, and improving operational efficiency in cleanroom environments. This paper presents a technical description of the European Space Agency's (ESA) project "AI for AR in Satellite AIT," which combines real-time computer vision and AR systems to assist technicians during satellite assembly. Leveraging Microsoft HoloLens 2 as the AR interface, the system delivers context-aware instructions and real-time feedback, tackling the complexities of object recognition and 6D pose estimation in AIT workflows. All AI models demonstrated over 70% accuracy, with the detection model exceeding 95% accuracy, indicating a high level of performance and reliability. A key contribution of this work lies in the effective use of synthetic data for training AI models in AR applications, addressing the significant challenges of obtaining real-world datasets in highly dynamic satellite environments, as well as the creation of the Segmented Anything Model for Automatic Labelling (SAMAL), which facilitates the automatic annotation of real data, achieving speeds up to 20 times faster than manual human annotation. The findings demonstrate the efficacy of AI-driven AR systems in automating critical satellite assembly tasks, setting a foundation for future innovations in the space industry.
9.0ROMar 10
Lightweight 3D LiDAR-Based UAV Tracking: An Adaptive Extended Kalman Filtering ApproachNivand Khosravi, Meysam Basiri, Rodrigo Ventura
Accurate relative positioning is crucial for swarm aerial robotics, enabling coordinated flight and collision avoidance. Although vision-based tracking has been extensively studied, 3D LiDAR-based methods remain underutilized despite their robustness under varying lighting conditions. Existing systems often rely on bulky, power-intensive sensors, making them impractical for small UAVs with strict payload and energy constraints. This paper presents a lightweight LiDAR-based UAV tracking system incorporating an Adaptive Extended Kalman Filter (AEKF) framework. Our approach effectively addresses the challenges posed by sparse, noisy, and nonuniform point cloud data generated by non-repetitive scanning 3D LiDARs, ensuring reliable tracking while remaining suitable for small drones with strict payload constraints. Unlike conventional filtering techniques, the proposed method dynamically adjusts the noise covariance matrices using innovation and residual statistics, thereby enhancing tracking accuracy under real-world conditions. Additionally, a recovery mechanism ensures continuity of tracking during temporary detection failures caused by scattered LiDAR returns or occlusions. Experimental validation was performed using a Livox Mid-360 LiDAR mounted on a DJI F550 UAV in real-world flight scenarios. The proposed method demonstrated robust UAV tracking performance under sparse LiDAR returns and intermittent detections, consistently outperforming both standard Kalman filtering and particle filtering approaches during aggressive maneuvers. These results confirm that the framework enables reliable relative positioning in GPS-denied environments without the need for multi-sensor arrays or external infrastructure.
8.0ROMar 12
Unsupervised LiDAR-Based Multi-UAV Detection and Tracking Under Extreme SparsityNivand Khosravi, Rodrigo Ventura, Meysam Basiri
Non-repetitive solid-state LiDAR scanning leads to an extremely sparse measurement regime for detecting airborne UAVs: a small quadrotor at 10-25 m typically produces only 1-2 returns per scan, which is far below the point densities assumed by most existing detection approaches and inadequate for robust multi-target data association. We introduce an unsupervised, LiDAR-only pipeline that addresses both detection and tracking without the need for labeled training data. The detector integrates range-adaptive DBSCAN clustering with a three-stage temporal consistency check and is benchmarked on real-world air-to-air flight data under eight different parameter configurations. The best setup attains 0.891 precision, 0.804 recall, and 0.63 m RMSE, and a systematic minPts sweep verifies that most scans contain at most 1-2 target points, directly quantifying the sparsity regime. For multi-target tracking, we compare deterministic Hungarian assignment with joint probabilistic data association (JPDA), each coupled with Interacting Multiple Model filtering, in four simulated scenarios with increasing levels of ambiguity. JPDA cuts identity switches by 64% with negligible impact on MOTA, demonstrating that probabilistic association is advantageous when UAV trajectories approach one another closely. A two-environment evaluation strategy, combining real-world detection with RTK-GPS ground truth and simulation-based tracking with identity-annotated ground truth, overcomes the limitations of GNSS-only evaluation at inter-UAV distances below 2 m.
6.2ROMar 11
Distributed Kalman--Consensus Filtering with Adaptive Uncertainty Weighting for Multi-Object Tracking in Mobile Robot NetworksNiusha Khosravi, Rodrigo Ventura, Meysam Basiri
This paper presents an implementation and evaluation of a Distributed Kalman--Consensus Filter (DKCF) for Multi-Object Tracking (MOT) in mobile robot networks operating under partial observability and heterogeneous localization uncertainty. A key challenge in such systems is the fusion of information from agents with differing localization quality, where frame misalignment can lead to inconsistent estimates, track duplication, and ghost tracks. To address this issue, we build upon the MOTLEE framework and retain its frame-alignment methodology, which uses consistently tracked dynamic objects as transient landmarks to improve relative pose estimates between robots. On top of this framework, we propose an uncertainty-aware adaptive consensus weighting mechanism that dynamically adjusts the influence of neighbor information based on the covariance of the transmitted estimates, thereby reducing the impact of unreliable data during distributed fusion. Local tracking is performed using a Kalman Filter (KF) with a Constant Velocity Model (CVM) and Global Nearest Neighbor (GNN) data association. simulation results demonstrate that adaptive weighting effectively protects local estimates from inconsistent data, yielding a MOTA improvement of 0.09 for agents suffering from localization drift, although system performance remains constrained by communication latency.
0.4ROMar 10
Robust Cooperative Localization in Featureless Environments: A Comparative Study of DCL, StCL, CCL, CI, and Standard-CLNivand Khosravi, Rodrigo Ventura, Meysam Basiri
Cooperative localization (CL) enables accurate position estimation in multi-robot systems operating in GPS-denied environments. This paper presents a comparative study of five CL approaches: Centralized Cooperative Localization (CCL), Decentralized Cooperative Localization (DCL), Sequential Cooperative Localization (StCL), Covariance Intersection (CI), and Standard Cooperative Localization (Standard-CL). All methods are implemented in ROS and evaluated through Monte Carlo simulations under two conditions: weak data association and robust detection. Our analysis reveals fundamental trade-offs among the methods. StCL and Standard-CL achieve the lowest position errors but exhibit severe filter inconsistency, making them unsuitable for safety-critical applications. DCL demonstrates remarkable stability under challenging conditions due to its measurement stride mechanism, which provides implicit regularization against outliers. CI emerges as the most balanced approach, achieving near-optimal consistency while maintaining competitive accuracy. CCL provides theoretically optimal estimation but shows sensitivity to measurement outliers. These findings offer practical guidance for selecting CL algorithms based on application requirements.
CVAug 29, 2025Code
CAD2DMD-SET: Synthetic Generation Tool of Digital Measurement Device CAD Model Datasets for fine-tuning Large Vision-Language ModelsJoão Valente, Atabak Dehban, Rodrigo Ventura
Recent advancements in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities across various multimodal tasks. They continue, however, to struggle with trivial scenarios such as reading values from Digital Measurement Devices (DMDs), particularly in real-world conditions involving clutter, occlusions, extreme viewpoints, and motion blur; common in head-mounted cameras and Augmented Reality (AR) applications. Motivated by these limitations, this work introduces CAD2DMD-SET, a synthetic data generation tool designed to support visual question answering (VQA) tasks involving DMDs. By leveraging 3D CAD models, advanced rendering, and high-fidelity image composition, our tool produces diverse, VQA-labelled synthetic DMD datasets suitable for fine-tuning LVLMs. Additionally, we present DMDBench, a curated validation set of 1,000 annotated real-world images designed to evaluate model performance under practical constraints. Benchmarking three state-of-the-art LVLMs using Average Normalised Levenshtein Similarity (ANLS) and further fine-tuning LoRA's of these models with CAD2DMD-SET's generated dataset yielded substantial improvements, with InternVL showcasing a score increase of 200% without degrading on other tasks. This demonstrates that the CAD2DMD-SET training dataset substantially improves the robustness and performance of LVLMs when operating under the previously stated challenging conditions. The CAD2DMD-SET tool is expected to be released as open-source once the final version of this manuscript is prepared, allowing the community to add different measurement devices and generate their own datasets.
LGJan 28
Conditional Denoising Model as a Physical Surrogate ModelJosé Afonso, Pedro Viegas, Rodrigo Ventura et al.
Surrogate modeling for complex physical systems typically faces a trade-off between data-fitting accuracy and physical consistency. Physics-consistent approaches typically treat physical laws as soft constraints within the loss function, a strategy that frequently fails to guarantee strict adherence to the governing equations, or rely on post-processing corrections that do not intrinsically learn the underlying solution geometry. To address these limitations, we introduce the {Conditional Denoising Model (CDM)}, a generative model designed to learn the geometry of the physical manifold itself. By training the network to restore clean states from noisy ones, the model learns a vector field that points continuously towards the valid solution subspace. We introduce a time-independent formulation that transforms inference into a deterministic fixed-point iteration, effectively projecting noisy approximations onto the equilibrium manifold. Validated on a low-temperature plasma physics and chemistry benchmark, the CDM achieves higher parameter and data efficiency than physics-consistent baselines. Crucially, we demonstrate that the denoising objective acts as a powerful implicit regularizer: despite never seeing the governing equations during training, the model adheres to physical constraints more strictly than baselines trained with explicit physics losses.
CVAug 29, 2025
FLORA: Efficient Synthetic Data Generation for Object Detection in Low-Data Regimes via finetuning Flux LoRAAlvaro Patricio, Atabak Dehban, Rodrigo Ventura
Recent advances in diffusion-based generative models have demonstrated significant potential in augmenting scarce datasets for object detection tasks. Nevertheless, most recent models rely on resource-intensive full fine-tuning of large-scale diffusion models, requiring enterprise-grade GPUs (e.g., NVIDIA V100) and thousands of synthetic images. To address these limitations, we propose Flux LoRA Augmentation (FLORA), a lightweight synthetic data generation pipeline. Our approach uses the Flux 1.1 Dev diffusion model, fine-tuned exclusively through Low-Rank Adaptation (LoRA). This dramatically reduces computational requirements, enabling synthetic dataset generation with a consumer-grade GPU (e.g., NVIDIA RTX 4090). We empirically evaluate our approach on seven diverse object detection datasets. Our results demonstrate that training object detectors with just 500 synthetic images generated by our approach yields superior detection performance compared to models trained on 5000 synthetic images from the ODGEN baseline, achieving improvements of up to 21.3% in mAP@.50:.95. This work demonstrates that it is possible to surpass state-of-the-art performance with far greater efficiency, as FLORA achieves superior results using only 10% of the data and a fraction of the computational cost. This work demonstrates that a quality and efficiency-focused approach is more effective than brute-force generation, making advanced synthetic data creation more practical and accessible for real-world scenarios.
LGFeb 11, 2025
Physics-consistent machine learning: output projection onto physical manifoldsMatilde Valente, Tiago C. Dias, Vasco Guerra et al.
Data-driven machine learning models often require extensive datasets, which can be costly or inaccessible, and their predictions may fail to comply with established physical laws. Current approaches for incorporating physical priors mitigate these issues by penalizing deviations from known physical laws, as in physics-informed neural networks, or by designing architectures that automatically satisfy specific invariants. However, penalization approaches do not guarantee compliance with physical constraints for unseen inputs, and invariant-based methods lack flexibility and generality. We propose a novel physics-consistent machine learning method that directly enforces compliance with physical principles by projecting model outputs onto the manifold defined by these laws. This procedure ensures that predictions inherently adhere to the chosen physical constraints, improving reliability and interpretability. Our method is demonstrated on two systems: a spring-mass system and a low-temperature reactive plasma. Compared to purely data-driven models, our approach significantly reduces errors in physical law compliance, enhances predictive accuracy of physical quantities, and outperforms alternatives when working with simpler models or limited datasets. The proposed projection-based technique is versatile and can function independently or in conjunction with existing physics-informed neural networks, offering a powerful, general, and scalable solution for developing fast and reliable surrogate models of complex physical systems, particularly in resource-constrained scenarios.
LGJan 20, 2022
Symplectic Momentum Neural Networks -- Using Discrete Variational Mechanics as a prior in Deep LearningSaul Santos, Monica Ekal, Rodrigo Ventura
With deep learning gaining attention from the research community for prediction and control of real physical systems, learning important representations is becoming now more than ever mandatory. It is of extreme importance that deep learning representations are coherent with physics. When learning from discrete data this can be guaranteed by including some sort of prior into the learning, however, not all discretization priors preserve important structures from the physics. In this paper, we introduce Symplectic Momentum Neural Networks (SyMo) as models from a discrete formulation of mechanics for non-separable mechanical systems. The combination of such formulation leads SyMos to be constrained towards preserving important geometric structures such as momentum and a symplectic form and learn from limited data. Furthermore, it allows to learn dynamics only from the poses as training data. We extend SyMos to include variational integrators within the learning framework by developing an implicit root-find layer which leads to End-to-End Symplectic Momentum Neural Networks (E2E-SyMo). Through experimental results, using the pendulum and cartpole, we show that such combination not only allows these models to learn from limited data but also provides the models with the capability of preserving the symplectic form and show better long-term behaviour.
RODec 11, 2021
Online Information-Aware Motion Planning with Inertial Parameter Learning for Robotic Free-FlyersMonica Ekal, Keenan Albee, Brian Coltin et al.
Space free-flyers like the Astrobee robots currently operating aboard the International Space Station must operate with inherent system uncertainties. Parametric uncertainties like mass and moment of inertia are especially important to quantify in these safety-critical space systems and can change in scenarios such as on-orbit cargo movement, where unknown grappled payloads significantly change the system dynamics. Cautiously learning these uncertainties en route can potentially avoid time- and fuel-consuming pure system identification maneuvers. Recognizing this, this work proposes RATTLE, an online information-aware motion planning algorithm that explicitly weights parametric model-learning coupled with real-time replanning capability that can take advantage of improved system models. The method consists of a two-tiered (global and local) planner, a low-level model predictive controller, and an online parameter estimator that produces estimates of the robot's inertial properties for more informed control and replanning on-the-fly; all levels of the planning and control feature online update-able models. Simulation results of RATTLE for the Astrobee free-flyer grappling an uncertain payload are presented alongside results of a hardware demonstration showcasing the ability to explicitly encourage model parametric learning while achieving otherwise useful motion.
OCDec 8, 2021
COSMIC: fast closed-form identification from large-scale data for LTV systemsMaria Carvalho, Claudia Soares, Pedro Lourenço et al.
We introduce a closed-form method for identification of discrete-time linear time-variant systems from data, formulating the learning problem as a regularized least squares problem where the regularizer favors smooth solutions within a trajectory. We develop a closed-form algorithm with guarantees of optimality and with a complexity that increases linearly with the number of instants considered per trajectory. The COSMIC algorithm achieves the desired result even in the presence of large volumes of data. Our method solved the problem using two orders of magnitude less computational power than a general purpose convex solver and was about 3 times faster than a Stochastic Block Coordinate Descent especially designed method. Computational times of our method remained in the order of magnitude of the second even for 10k and 100k time instants, where the general purpose solver crashed. To prove its applicability to real world systems, we test with spring-mass-damper system and use the estimated model to find the optimal control path. Our algorithm was applied to both a Low Fidelity and Functional Engineering Simulators for the Comet Interceptor mission, that requires precise pointing of the on-board cameras in a fast dynamics environment. Thus, this paper provides a fast alternative to classical system identification techniques for linear time-variant systems, while proving to be a solid base for applications in the Space industry and a step forward to the incorporation of algorithms that leverage data in such a safety-critical environment.
LGNov 2, 2021
Decision Support Models for Predicting and Explaining Airport Passenger Connectivity from DataMarta Guimaraes, Claudia Soares, Rodrigo Ventura
Predicting if passengers in a connecting flight will lose their connection is paramount for airline profitability. We present novel machine learning-based decision support models for the different stages of connection flight management, namely for strategic, pre-tactical, tactical and post-operations. We predict missed flight connections in an airline's hub airport using historical data on flights and passengers, and analyse the factors that contribute additively to the predicted outcome for each decision horizon. Our data is high-dimensional, heterogeneous, imbalanced and noisy, and does not inform about passenger arrival/departure transit time. We employ probabilistic encoding of categorical classes, data balancing with Gaussian Mixture Models, and boosting. For all planning horizons, our models attain an AUC of the ROC higher than 0.93. SHAP value explanations of our models indicate that scheduled/perceived connection times contribute the most to the prediction, followed by passenger age and whether border controls are required.
IMMay 14, 2021
Conjunction Data Messages behave as a Poisson ProcessFrancisco Caldas, Claudia Soares, Cláudia Nunes et al.
Space debris is a major problem in space exploration. International bodies continuously monitor a large database of orbiting objects and emit warnings in the form of conjunction data messages. An important question for satellite operators is to estimate when fresh information will arrive so that they can react timely but sparingly with satellite maneuvers. We propose a statistical learning model of the message arrival process, allowing us to answer two important questions: (1) Will there be any new message in the next specified time interval? (2) When exactly and with what uncertainty will the next message arrive? The average prediction error for question (2) of our Bayesian Poisson process model is smaller than the baseline in more than 4 hours in a test set of 50k close encounter events.
ROFeb 20, 2021
Safe and Uncertainty-Aware Robotic Motion Planning Techniques for Agile On-Orbit AssemblyBryce Doerr, Keenan Albee, Monica Ekal et al.
As access to space and robotic autonomy capabilities move forward, there is simultaneously a growing interest in deploying large, complex space structures to provide new on-orbit capabilities. New space-borne observatories, large orbital outposts, and even futuristic on-orbit manufacturing will be enabled by robotic assembly of space structures using techniques like on-orbit additive manufacturing which can provide flexibility in constructing and even repairing complex hardware. However, the dynamics underlying the robotic assembler during manipulation may operate under inertial uncertainties. Thus, inertial estimation of the robot and the manipulated component system must be considered during structural assembly. The contribution of this work is to address both the motion planning and control for robotic assembly with consideration of the inertial estimation of the combined free-flying robotic assembler and additively manufactured component system. Specifically, the Linear Quadratic Regulator Rapidly-Exploring Randomized Trees (LQR-RRT*) and dynamically feasible path smoothing are used to obtain obstacle-free trajectories for the system. Further, model learning is incorporated explicitly into the planning stages via approximation of the continuous system and accompanying reward of performing safe, objective-oriented motion. Remaining uncertainty can then be dealt with using robust tube model predictive control. By obtaining controlled trajectories that consider both obstacle avoidance and learning of the inertial properties of the free-flyer and manipulated component system, the free-flyer rapidly considers and plans the construction of space structures with enhanced system knowledge. The approach naturally generalizes to repairing, refueling, and re-provisioning space structure components while providing optimal collision-free trajectories under e.g., inertial uncertainty.
ROMay 12, 2020
Human-Robot Collaboration in Microgravity: the Object Handover ProblemAdriana Fernandes, Rodrigo Ventura
Collaborative space robots are an emerging technology with high impact as robots facilitate servicing functions in collaboration with astronauts with higher precision during lengthy tasks, under tight operational schedules, with less risk and costs, making them more efficient and economically more viable. However, human-robot collaboration in space is still a challenge concerning key issues in human-robot interaction, including mobility and collaborative manipulation of objects on a microgravity environment. In this paper we formulate an algorithm that enables a free-flyer robot, equipped with a manipulator, to perform an object handover between a human and a free-flyer robot, in a microgravity environment. To validate and evaluate this algorithm, we present a systematic user study with the goal of understanding the subjective outcome effects of a rigid and compliant impedance robot behavior during the interaction. The results showed that the rigid behavior was overall more preferable and registered higher transfer success during the tasks.
ROMar 20, 2020
Robust Plan Execution with Unexpected ObservationsOscar Lima, Michael Cashmore, Daniele Magazzeni et al.
In order to ensure the robust actuation of a plan, execution must be adaptable to unexpected situations in the world and to exogenous events. This is critical in domains in which committing to a wrong ordering of actions can cause the plan failure, even when all the actions succeed. We propose an approach to the execution of a task plan that permits some adaptability to unexpected observations of the state while maintaining the validity of the plan through online reasoning. Our approach computes an adaptable, partially-ordered plan from a given totally-ordered plan. The partially-ordered plan is adaptable in that it can exploit beneficial differences between the world and what was expected. The approach is general in that it can be used with any task planner that produces either a totally or a partially-ordered plan. We propose a plan execution algorithm that computes online the complete set of valid totally-ordered plans described by an adaptable partially-ordered plan together with the probability of success for each of them. This set is then used to choose the next action to execute.
SYDec 20, 2019
Teaching robots to perceive time -- A reinforcement learning approach (Extended version)Inês Lourenço, Bo Wahlberg, Rodrigo Ventura
Time perception is the phenomenological experience of time by an individual. In this paper, we study how to replicate neural mechanisms involved in time perception, allowing robots to take a step towards temporal cognition. Our framework follows a twofold biologically inspired approach. The first step consists of estimating the passage of time from sensor measurements, since environmental stimuli influence the perception of time. Sensor data is modeled as Gaussian processes that represent the second-order statistics of the natural environment. The estimated elapsed time between two events is computed from the maximum likelihood estimate of the joint distribution of the data collected between them. Moreover, exactly how time is encoded in the brain remains unknown, but there is strong evidence of the involvement of dopaminergic neurons in timing mechanisms. Since their phasic activity has a similar behavior to the reward prediction error of temporal-difference learning models, the latter are used to replicate this behavior. The second step of this approach consists therefore of applying the agent's estimate of the elapsed time in a reinforcement learning problem, where a feature representation called Microstimuli is used. We validate our framework by applying it to an experiment that was originally conducted with mice, and conclude that a robot using this framework is able to reproduce the timing mechanisms of the animal's brain.
ROJun 6, 2019
Combining Parameter Identification and Trajectory Optimization: Real-time Planning for Information GainKeenan Albee, Monica Ekal, Rodrigo Ventura et al.
Robotic systems often operate with uncertainties in their dynamics, for example, unknown inertial properties. Broadly, there are two approaches for controlling uncertain systems: design robust controllers in spite of uncertainty, or characterize a system before attempting to control it. This paper proposes a middle-ground approach, making trajectory progress while also accounting for gaining information about the system. More specifically, it combines excitation trajectories which are usually intended to optimize information gain for an estimator, with goal-driven trajectory optimization metrics. For this purpose, a measure of information gain is incorporated (using the Fisher Information Matrix) in a real-time planning framework to produce trajectories favorable for estimation. At the same time, the planner receives stable parameter updates from the estimator, enhancing the system model. An implementation of this learn-as-you-go approach utilizing an Unscented Kalman Filter (UKF) and Nonlinear Model Predictive Controller (NMPC) is demonstrated in simulation. Results for cases with and without information gain and online parameter updates in the system model are presented.
SYApr 7, 2019
Nonlinear Model Predictive Control for 3D Formation of Multirotor Micro Aerial Vehicles with Relative Sensing in Local CoordinatesI. Kagan Erunsal, Rodrigo Ventura, Alcherio Martinoli
The complex tasks such as surveillance, construction, search and rescue can benefit of the maneuverability of multirotor Micro Aerial Vehicles (MAVs) to obtain robust, cooperative system behavior and formation control is a prominent component of the these complex tasks. This work focuses on the problem of three-dimensional formation control of multirotor MAVs by using exclusively relative sensory information. It proposes a centralized Nonlinear Model Predictive Control (NMPC) approach in a leader-follower scheme. A realistic six degrees of freedom mathematical model of a multirotor MAVs is introduced and leveraged in the control laws. The formulation of the problem is performed based on NMPC and relative sensing framework with respect to local coordinate frames of the robots. This type of formulation makes the formation independent of the full knowledge of global or common reference frames and the utilization of expensive global localization sensors. Real-time Iteration (RTI) based solution to optimal control problem (OCP) is proposed by taking the novel formulation into account. An extensive scenario is designed to test and validate the strategy. Evaluation of the results suggests that satisfactory robust performance is achieved and maintained under model uncertainty and noise in local sensors and even in cases where the dynamics of the formation suddenly changes.
CLJul 9, 2018
A deep learning approach for understanding natural language commands for mobile service robotsPedro Henrique Martins, Luís Custódio, Rodrigo Ventura
Using natural language to give instructions to robots is challenging, since natural language understanding is still largely an open problem. In this paper we address this problem by restricting our attention to commands modeled as one action, plus arguments (also known as slots). For action detection (also called intent detection) and slot filling various architectures of Recurrent Neural Networks and Long Short Term Memory (LSTM) networks were evaluated, having LSTMs achieved a superior accuracy. As the action requested may not fall within the robots capabilities, a Support Vector Machine(SVM) is used to determine whether it is or not. For the input of the neural networks, several word embedding algorithms were compared. Finally, to implement the system in a robot, a ROS package is created using a SMACH state machine. The proposed system is then evaluated both using well-known datasets and benchmarks in the context of domestic service robots.
ROFeb 26, 2018
An Energy Balance Based Method for Parameter Identification of a Free-Flying Robot Grasping An Unknown ObjectMonica Ekal, Rodrigo Ventura
The estimation of inertial parameters of a robotic system is crucial for better trajectory tracking performance, specially when model-based controllers are used for carrying out precise tasks. In this paper, we consider the scenario of grasping an object of unknown properties by a free-flyer space robot with limited actuation. The problem is to find the inertial parameters of the complete system after grasping has been performed. Excitation is provided in inertial space, and the excitation trajectories are found by optimization. Truncated Fourier series are used to represent the reference as well as tracked trajectory. An approach based on the energy balance between the actuation work and the rate of change of kinetic energy is introduced to calculate the number of harmonics in the Fourier series used to represent the executed trajectory, while trying to find a balance between accounting for saturation effects and keeping out noise. The effect of input saturation on parameter estimation is also studied. Simulation results using the Space CoBot free-flyer robot are presented to show the feasibility of the approach.
ROMar 24, 2016
Space CoBot: modular design of an holonomic aerial robot for indoor microgravity environmentsPedro Roque, Rodrigo Ventura
This paper presents the design of a small aerial robot for inhabited microgravity environments, such as orbiting space stations (e.g., ISS). In particular, we target a fleet of robots, called Space CoBots, for collaborative tasks with humans, such as telepresence and cooperative mobile manipulation. The design is modular, comprising an hexrotor based propulsion system, and a stack of modules including batteries, cameras for navigation, a screen for telepresence, a robotic arm, space for extension modules, and a pair of docking ports. These ports can be used for docking and for mechanically attaching two Space CoBots together. The kinematics is holonomic, and thus the translational and the rotational components can be fully decoupled. We employ a multi-criteria optimization approach to determine the best geometric configuration for maximum thrust and torque across all directions. We also tackle the problem of motion control: we use separate converging controllers for position and attitude control. Finally, we present simulation results using a realistic physics simulator. These experiments include a sensitivity evaluation to sensor noise and to unmodeled dynamics, namely a load transportation.
ROFeb 10, 2016
Space CoBot: a collaborative aerial robot for indoor microgravity environmentsPedro Roque, Rodrigo Ventura
This paper presents a first contribution to the design of a small aerial robot for inhabited microgravity environments, such as orbiting space stations. In particular, we target a fleet of robots for collaborative tasks with humans, such as telepresence and cooperative mobile manipulation. We explore a propeller based propulsion system, arranged in such a way that the translational and the rotational components can be decoupled, resulting in an holonomic hexarotor. Since propellers have limited thrust, we employ an optimization approach to select the geometric configuration given a criteria of uniform maximum thrust across all directions in the body reference frame. We also tackle the problem of motion control: due to the decoupling of translational and rotational modes we use separate converging controllers for each one of these modes. In addition, we present preliminary simulation results in a realistic simulator, in closed loop with the proposed controller, thus providing a first validation of the followed methodology.