Stjepan Bogdan

Juraj Obradovic, Marko Krizmancic, Stjepan Bogdan

This paper presents a decentralized leader-follower multi-robot formation control based on a reinforcement learning (RL) algorithm applied to a swarm of small educational Sphero robots. Since the basic Q-learning method is known to require large memory resources for Q-tables, this work implements the Double Deep Q-Network (DDQN) algorithm, which has achieved excellent results in many robotic problems. To enhance the system behavior, we trained two different DDQN models, one for reaching the formation and the other for maintaining it. The models use a discrete set of robot motions (actions) to adapt the continuous nonlinear system to the discrete nature of RL. The presented approach has been tested in simulation and real experiments which show that the multi-robot system can achieve and maintain a stable formation without the need for complex mathematical models and nonlinear control laws.

Antonella Barisic Kulas, Andreja Jurasovic, Stjepan Bogdan

Thermal imaging from unmanned aerial vehicles (UAVs) holds significant potential for applications in search and rescue, wildlife monitoring, and emergency response, especially under low-light or obscured conditions. However, the scarcity of large-scale, diverse thermal aerial datasets limits the advancement of deep learning models in this domain, primarily due to the high cost and logistical challenges of collecting thermal data. In this work, we introduce a novel procedural pipeline for generating synthetic thermal images from an aerial perspective. Our method integrates arbitrary object classes into existing thermal backgrounds by providing control over the position, scale, and orientation of the new objects, while aligning them with the viewpoints of the background. We enhance existing thermal datasets by introducing new object categories, specifically adding a drone class in urban environments to the HIT-UAV dataset and an animal category to the MONET dataset. In evaluating these datasets for object detection task, we showcase strong performance across both new and existing classes, validating the successful expansion into new applications. Through comparative analysis, we show that thermal detectors outperform their visible-light-trained counterparts and highlight the importance of replicating aerial viewing angles. Project page: https://github.com/larics/thermal_aerial_synthetic.

Martin Křížek, Matouš Vrba, Antonella Barišić Kulaš et al.

We propose a new approach to visual perception for relative localization of agents within large-scale swarms of UAVs. Inspired by biological perception utilized by schools of sardines, swarms of bees, and other large groups of animals capable of moving in a decentralized yet coherent manner, our method does not rely on detecting individual neighbors by each agent and estimating their relative position, but rather we propose to regress a neighbor density over distance. This allows for a more accurate distance estimation as well as better scalability with respect to the number of neighbors. Additionally, a novel swarm control algorithm is proposed to make it compatible with the new relative localization method. We provide a thorough evaluation of the presented methods and demonstrate that the regressing approach to distance estimation is more robust to varying relative pose of the targets and that it is suitable to be used as the main source of relative localization for swarm stabilization.

Anja Bosak, Dorian Erić, Ana Milas et al.

In this paper, we present a generalized, comprehensive nonlinear mathematical model and conceptual design for the MetaMorpher, a metamorphic Unmanned Aerial Vehicle (UAV) designed to bridge the gap between vertical takeoff and landing agility and fixed-wing cruising efficiency. Building on the successful design of the spincopter platform, this work introduces a simplified mechanical architecture using lightweight materials and a novel wing-folding strategy. Unlike traditional rigid-body approximations, we derive a nonlinear flight dynamics model that enables arbitrary force distributions across a segmented wing structure. This modularity allows for testing different airfoils, mass distributions, and chord lengths in a single environment. As part of this work, various flight modes were specifically tested and analyzed in the Simulink environment. The results show that the model behaves predictably under different structural configurations, demonstrating its reliability as a tool for rapid design evaluation.

Antonella Barisic Kulas, Frano Petric, Stjepan Bogdan

Autonomous maritime surveillance and target vessel identification in environments where Global Navigation Satellite Systems (GNSS) are not available is critical for a number of applications such as search and rescue and threat detection. When the target vessel is only described by visual cues and its last known position is not available, unmanned aerial vehicles (UAVs) must rely solely on on-board vision to scan a large search area under strict computational constraints. To address this challenge, we leverage the YOLOv8 object detection model to detect all vessels in the field of view. We then apply feature matching and hue histogram distance analysis to determine whether any detected vessel corresponds to the target. When found, we localize the target using simple geometric principles. We demonstrate the proposed method in real-world experiments during the MBZIRC2023 competition, integrated into a fully autonomous system with GNSS-denied navigation. We also evaluate the impact of perspective on detection accuracy and localization precision and compare it with the oracle approach.

Ivo Vatavuk, Marsela Polić, Ivan Hrabar et al.

In this paper we present our hardware design and control approaches for a mobile manipulation platform used in Challenge 2 of the MBZIRC 2020 competition. In this challenge, a team of UAVs and a single UGV collaborate in an autonomous, wall-building scenario, motivated by construction automation and large-scale robotic 3D printing. The robots must be able, autonomously, to detect, manipulate, and transport bricks in an unstructured, outdoor environment. Our control approach is based on a state machine that dictates which controllers are active at each stage of the Challenge. In the first stage our UGV uses visual servoing and local controllers to approach the target object without considering its orientation. The second stage consists of detecting the object's global pose using OpenCV-based processing of RGB-D image and point-cloud data, and calculating an alignment goal within a global map. The map is built with Google Cartographer and is based on onboard LIDAR, IMU, and GPS data. Motion control in the second stage is realized using the ROS Move Base package with Time-Elastic Band trajectory optimization. Visual servo algorithms guide the vehicle in local object-approach movement and the arm in manipulating bricks. To ensure a stable grasp of the brick's magnetic patch, we developed a passively-compliant, electromagnetic gripper with tactile feedback. Our fully-autonomous UGV performed well in Challenge 2 and in post-competition evaluations of its brick pick-and-place algorithms.

Antonella Barisic, Frano Petric, Stjepan Bogdan

In this paper we propose a novel approach to generate a synthetic aerial dataset for application in UAV monitoring. We propose to accentuate shape-based object representation by applying texture randomization. A diverse dataset with photorealism in all parameters such as shape, pose, lighting, scale, viewpoint, etc. except for atypical textures is created in a 3D modelling software Blender. Our approach specifically targets two conditions in aerial images where texture of objects is difficult to detect, namely challenging illumination and objects occupying only a small portion of the image. Experimental evaluation of YOLO and Faster R-CNN detectors trained on synthetic data with randomized textures confirmed our approach by increasing the mAP value (17 and 3.7 percentage points for YOLO; 20 and 1.1 percentage points for Faster R-CNN) on two test datasets of real images, both containing UAV-to-UAV images with motion blur. Testing on different domains, we conclude that the more the generalisation ability is put to the test, the more apparent are the advantages of the shape-based representation.

Barbara Arbanas Ferreira, Tamara Petrović, Stjepan Bogdan

In this paper, we propose a distributed multi-stage optimization method for planning complex missions for heterogeneous multi-robot teams. This class of problems involves tasks that can be executed in different ways and are associated with cross-schedule dependencies that constrain the schedules of the different robots in the system. The proposed approach involves a multi-objective heuristic search of the mission, represented as a hierarchical tree that defines the mission goal. This procedure outputs several favorable ways to fulfill the mission, which directly feed into the next stage of the method. We propose a distributed metaheuristic based on evolutionary computation to allocate tasks and generate schedules for the set of chosen decompositions. The method is evaluated in a simulation setup of an automated greenhouse use case, where we demonstrate the method's ability to adapt the planning strategy depending on the available robots and the given optimization criteria.

Ana Batinovic, Antun Ivanovic, Tamara Petrovic et al.

In this paper, we address the problem of autonomous exploration of unknown environments with an aerial robot equipped with a sensory set that produces large point clouds, such as LiDARs. The main goal is to gradually explore an area while planning paths and calculating information gain in short computation time, suitable for implementation on an on-board computer. To this end, we present a planner that randomly samples viewpoints in the environment map. It relies on a novel and efficient gain calculation based on the Recursive Shadowcasting algorithm. To determine the Next-Best-View (NBV), our planner uses a cuboid-based evaluation method that results in an enviably short computation time. To reduce the overall exploration time, we also use a dead end resolving strategy that allows us to quickly recover from dead ends in a challenging environment. Comparative experiments in simulation have shown that our approach outperforms the current state-of-the-art in terms of computational efficiency and total exploration time. The video of our approach can be found at https://www.youtube.com/playlist?list=PLC0C6uwoEQ8ZDhny1VdmFXLeTQOSBibQl.

Lovro Markovic, Marin Kovac, Robert Milijas et al.

This paper addresses the issues of unmanned aerial vehicle (UAV) indoor navigation, specifically in areas where GPS and magnetometer sensor measurements are unavailable or unreliable. The proposed solution is to use an error state extended Kalman filter (ES -EKF) in the context of multi-sensor fusion. Its implementation is adapted to fuse measurements from multiple sensor sources and the state model is extended to account for sensor drift and possible calibration inaccuracies. Experimental validation is performed by fusing IMU data obtained from the PixHawk 2.1 flight controller with pose measurements from LiDAR Cartographer SLAM, visual odometry provided by the Intel T265 camera and position measurements from the Pozyx UWB indoor positioning system. The estimated odometry from ES-EKF is validated against ground truth data from the Optitrack motion capture system and its use in a position control loop to stabilize the UAV is demonstrated.

Barbara Arbanas Ferreira, Tamara Petrović, Matko Orsag et al.

To enable safe and efficient use of multi-robot systems in everyday life, a robust and fast method for coordinating their actions must be developed. In this paper, we present a distributed task allocation and scheduling algorithm for missions where the tasks of different robots are tightly coupled with temporal and precedence constraints. The approach is based on representing the problem as a variant of the vehicle routing problem, and the solution is found using a distributed metaheuristic algorithm based on evolutionary computation (CBM-pop). Such an approach allows a fast and near-optimal allocation and can therefore be used for online replanning in case of task changes. Simulation results show that the approach has better computational speed and scalability without loss of optimality compared to the state-of-the-art distributed methods. An application of the planning procedure to a practical use case of a greenhouse maintained by a multi-robot system is given.

Antonella Barisic, Frano Petric, Stjepan Bogdan

This paper presents our approach to intercepting a faster intruder UAV, inspired by the MBZIRC 2020 Challenge 1. By utilizing a priori knowledge of the shape of the intruder's trajectory, we can calculate an interception point. Target tracking is based on image processing by a YOLOv3 Tiny convolutional neural network, combined with depth calculation using a gimbal-mounted ZED Mini stereo camera. We use RGB and depth data from the camera, devising a noise-reducing histogram-filter to extract the target's 3D position. Obtained 3D measurements of target's position are used to calculate the position, orientation, and size of a figure-eight shaped trajectory, which we approximate using a Bernoulli lemniscate. Once the approximation is deemed sufficiently precise, as measured by the distance between observations and estimate, we calculate an interception point to position the interceptor UAV directly on the intruder's path. Our method, which we have significantly improved based on the experience gathered during the MBZIRC competition, has been validated in simulation and through field experiments. Our results confirm that we have developed an efficient, visual-perception module that can extract information describing the intruder UAV's motion with precision sufficient to support interception planning. In a majority of our simulated encounters, we can track and intercept a target that moves 30% faster than the interceptor. Corresponding tests in an unstructured environment yielded 9 out of 12 successful results.

Juraj Orsulić, Robert Milijas, Ana Batinovic et al.

This paper describes an application of the Cartographer graph SLAM stack as a pose sensor in a UAV feedback control loop, with certain application-specific changes in the SLAM stack such as smoothing of the optimized pose. Pose estimation is performed by fusing 3D LiDAR/IMU-based proprioception with GPS position measurements by means of pose graph optimisation. Moreover, partial environment maps built from the LiDAR data (submaps) within the Cartographer SLAM stack are marshalled into OctoMap, an Octree-based voxel map implementation. The OctoMap is further used for navigation tasks such as path planning and obstacle avoidance.

Robert Milijas, Lovro Markovic, Antun Ivanovic et al.

This paper investigates the use of LiDAR SLAM as a pose feedback for autonomous flight. Cartographer, LOAM and HDL graph SLAM are first introduced on a conceptual level and later tested for this role. They are first compared offline on a series of datasets to see if they are capable of producing high-quality pose estimates in agile and long-range flight scenarios. The second stage of testing consists of integrating the SLAM algorithms into a cascade PID UAV control system and comparing the control system performance on step excitation signals and helical trajectories. The comparison is based on step response characteristics and several time integral performancecriteria as well as the RMS error between planned and executed trajectory.

Ana Batinović, Tamara Petrović, Antun Ivanovic et al.

In this paper we propose a planner for 3D exploration that is suitable for applications using state-of-the-art 3D sensors such as lidars, which produce large point clouds with each scan. The planner is based on the detection of a frontier - a boundary between the explored and unknown part of the environment - and consists of the algorithm for detecting frontier points, followed by clustering of frontier points and selecting the best frontier point to be explored. Compared to existing frontier-based approaches, the planner is more scalable, i.e. it requires less time for the same data set size while ensuring similar exploration time. Performance is achieved by not relying on data obtained directly from the 3D sensor, but on data obtained by a mapping algorithm. In order to cluster the frontier points, we use the properties of the Octree environment representation, which allows easy analysis with different resolutions. The planner is tested in the simulation environment and in an outdoor test area with a UAV equipped with a lidar sensor. The results show the advantages of the approach.