David St-Onge

Abdalwhab Abdalwhab, Giovanni Beltrame, Samira Ebrahimi Kahou et al.

Robotics can help address the growing worker shortage challenge of the manufacturing industry. As such, machine tending is a task collaborative robots can tackle that can also highly boost productivity. Nevertheless, existing robotics systems deployed in that sector rely on a fixed single-arm setup, whereas mobile robots can provide more flexibility and scalability. In this work, we introduce a multi-agent multi-machine tending learning framework by mobile robots based on Multi-agent Reinforcement Learning (MARL) techniques with the design of a suitable observation and reward. Moreover, an attention-based encoding mechanism is developed and integrated into Multi-agent Proximal Policy Optimization (MAPPO) algorithm to boost its performance for machine tending scenarios. Our model (AB-MAPPO) outperformed MAPPO in this new challenging scenario in terms of task success, safety, and resources utilization. Furthermore, we provided an extensive ablation study to support our various design decisions.

Alexandre Duval, Anita Paas, Abdalwhab Abdalwhab et al.

The drone industry is diversifying and the number of pilots increases rapidly. In this context, flight schools need adapted tools to train pilots, most importantly with regard to their own awareness of their physiological and cognitive limits. In civil and military aviation, pilots can train themselves on realistic simulators to tune their reaction and reflexes, but also to gather data on their piloting behavior and physiological states. It helps them to improve their performances. Opposed to cockpit scenarios, drone teleoperation is conducted outdoor in the field, thus with only limited potential from desktop simulation training. This work aims to provide a solution to gather pilots behavior out in the field and help them increase their performance. We combined advance object detection from a frontal camera to gaze and heart-rate variability measurements. We observed pilots and analyze their behavior over three flight challenges. We believe this tool can support pilots both in their training and in their regular flight tasks. A demonstration video is available on https://www.youtube.com/watch?v=eePhjd2qNiI

Megan Heath, Ali Imran, David St-Onge

Precision agriculture aims to use technological tools for the agro-food sector to increase productivity, cut labor costs, and reduce the use of resources. This work takes inspiration from bees vision to design a remote sensing system tailored to incorporate UV-reflectance into a flower detector. We demonstrate how this approach can provide feature-rich images for deep learning strawberry flower detection and we apply it to a scalable, yet cost effective aerial monitoring robotic system in the field. We also compare the performance of our UV-G-B image detector with a similar work that utilizes RGB images.

Ali Imran, Giovanni Beltrame, David St-Onge

In industrial environments, predicting human actions is essential for ensuring safe and effective collaboration between humans and robots. This paper introduces a perception framework that enables mobile robots to understand and share information about human actions in a decentralized way. The framework first allows each robot to build a spatial graph representing its surroundings, which it then shares with other robots. This shared spatial data is combined with temporal information to track human behavior over time. A swarm-inspired decision-making process is used to ensure all robots agree on a unified interpretation of the human's actions. Results show that adding more robots and incorporating longer time sequences improve prediction accuracy. Additionally, the consensus mechanism increases system resilience, making the multi-robot setup more reliable in dynamic industrial settings.

Abdalwhab Abdalwhab, Ali Imran, Sina Heydarian et al.

The construction industry has long explored robotics and computer vision, yet their deployment on construction sites remains very limited. These technologies have the potential to revolutionize traditional workflows by enhancing accuracy, efficiency, and safety in construction management. Ground robots equipped with advanced vision systems could automate tasks such as monitoring mechanical, electrical, and plumbing (MEP) systems. The present research evaluates the applicability of open-vocabulary vision-language models compared to fine-tuned, lightweight, closed-set object detectors for detecting MEP components using a mobile ground robotic platform. A dataset collected with cameras mounted on a ground robot was manually annotated and analyzed to compare model performance. The results demonstrate that, despite the versatility of vision-language models, fine-tuned lightweight models still largely outperform them in specialized environments and for domain-specific tasks.

Abdalwhab Abdalwhab, Giovanni Beltrame, David St-Onge

The manufacturing sector was recently affected by workforce shortages, a problem that automation and robotics can heavily minimize. Simultaneously, reinforcement learning (RL) offers a promising solution where robots can learn through interaction with the environment. In this work, we leveraged a simplistic robotic system to work with RL with "real" data without having to deploy large expensive robots in a manufacturing setting. A real-world tabletop arena was designed with robots that mimic the agents' behavior in the simulation. Despite the difference in dynamics and machine size, the robots were able to depict the same behavior as in the simulation. In addition, those experiments provided an initial understanding of the real deployment challenges.

Rafael Gomes Braga, Sina Karimi, Ulrich Dah-Achinanon et al.

Several deployment locations of mobile robotic systems are human made (i.e. urban firefighter, building inspection, property security) and the manager may have access to domain-specific knowledge about the place, which can provide semantic contextual information allowing better reasoning and decision making. In this paper we propose a system that allows a mobile robot to operate in a location-aware and operator-friendly way, by leveraging semantic information from the deployment location and integrating it to the robots localization and navigation systems. We integrate Building Information Models (BIM) into the Robotic Operating System (ROS), to generate topological and metric maps fed to an layered path planner (global and local). A map merging algorithm integrates newly discovered obstacles into the metric map, while a UWB-based localization system detects equipment to be registered back into the semantic database. The results are validated in simulation and real-life deployments in buildings and construction sites.

Sina Karimi, Rafael Gomes Braga, Ivanka Iordanova et al.

With the growth in automated data collection of construction projects, the need for semantic navigation of mobile robots is increasing. In this paper, we propose an infrastructure to leverage building-related information for smarter, safer and more precise robot navigation during construction phase. Our use of Building Information Models (BIM) in robot navigation is twofold: (1) the intuitive semantic information enables non-experts to deploy robots and (2) the semantic data exposed to the navigation system allows optimal path planning (not necessarily the shortest one). Our Building Information Robotic System (BIRS) uses Industry Foundation Classes (IFC) as the interoperable data format between BIM and the Robotic Operating System (ROS). BIRS generates topological and metric maps from BIM for ROS usage. An optimal path planer, integrating critical components for construction assessment is proposed using a cascade strategy (global versus local). The results are validated through series of experiments in construction sites.

Sina Karimi, Ivanka Iordanova, David St-Onge

With growth in the use of autonomous Unmanned Ground Vehicle (UGV) for automated data collection from construction projects, the problem of inter-disciplinary semantic data sharing and exchanges between construction and robotic domains has attracted construction stakeholders' attention. Cross-domain data translation requires detailed specifications especially when it comes to semantic data translation. Building Information Modeling (BIM) and Geographic Information System (GIS) are the two technologies to capture and store construction data for indoor structure and outdoor environment respectively. In the absence of a standard format for data exchanges between the construction and robotic domains, the tools of both industries are yet to be integrated in a coherent deployment infrastructure. Hence, the semantics of BIM-GIS cannot be automatically integrated by any robotic platform. To enable semantic data transfer across domains, semantic web technology has been widely used in multidisciplinary areas for interoperability. We exploit it to pave the way to a smarter, quicker and more precise robot navigation on job-sites. This paper develops a semantic web ontology integrating robot navigation and data collection to convey the meanings from BIM-GIS to the robot. The proposed Building Information Robotic System (BIRS) provides construction data that are semantically transferred to the robotic platform and can be used by the robot navigation software stack on construction sites. To reach this objective, we first need to bridge the knowledge representation between construction and robotic domains. Then, we develop a semantic database to integrate with Robot Operating System (ROS) which can communicate with the robot and the navigation system in order to provide the robot with semantic building data at each step of data collection. Finally, the proposed system is validated through a case study.

Hamed Montazer Zohour, Bruno Belzile, David St-Onge

A polynomial solution to the inverse kinematic problem of the Kinova Gen3 Lite robot is proposed in this paper. This serial robot is based on a 6R kinematic chain and is not wrist-partitioned. We first start from the forward kinematics equation providing the position and orientation of the end-effector, finally, the univariate polynomial equation is given as a function of the first joint variable $θ_{1}$. The remaining joint variables are computed by back substitution. Thus, an unique set of joint position is obtain for each root of the univariate equation. Numerical examples, simulated in ROS (Robot Operating System), are given to validate the results, which are compared to the coordinates obtained with MoveIt! and with the actual robot. A procedure to choose an optimum posture of the robot is also proposed.

Vivek Shankar Varadharajan, David St-Onge, Bram Adams et al.

The coordination of robot swarms - large decentralized teams of robots - generally relies on robust and efficient inter-robot communication. Maintaining communication between robots is particularly challenging in field deployments. Unstructured environments, limited computational resources, low bandwidth, and robot failures all contribute to the complexity of connectivity maintenance. In this paper, we propose a novel lightweight algorithm to navigate a group of robots in complex environments while maintaining connectivity by building a chain of robots. The algorithm is robust to single robot failures and can heal broken communication links. The algorithm works in 3D environments: when a region is unreachable by wheeled robots, the chain is extended with flying robots. We test the performance of the algorithm using up to 100 robots in a physics-based simulator with three mazes and different robot failure scenarios. We then validate the algorithm with physical platforms: 7 wheeled robots and 6 flying ones, in homogeneous and heterogeneous scenarios.

Jacopo Panerati, Benjamin Ramtoula, David St-Onge et al.

Redundancy and parallelism make decentralized multi-robot systems appealing solutions for the exploration of extreme environments. However, effective cooperation often requires team-wide connectivity and a carefully designed communication strategy. Several recently proposed decentralized connectivity maintenance approaches exploit elegant algebraic results drawn from spectral graph theory. Yet, these proposals are rarely taken beyond simulations or laboratory implementations. In this work, we present two major contributions: (i) we describe the full-stack implementation---from hardware to software---of a decentralized control law for robust connectivity maintenance; and (ii) we assess, in the field, our setup's ability to correctly exchange all the necessary information required to maintain connectivity in a team of quadcopters.

Yanjun Cao, David St-Onge, Andreas Zell et al.

Localization and tracking are two very active areas of research for robotics, automation, and the Internet-of-Things. Accurate tracking for a large number of devices usually requires deployment of substantial infrastructure (infrared tracking systems, cameras, wireless antennas, etc.), which is not ideal for inaccessible or protected environments. This paper stems from the challenge posed such environments: cover a large number of units spread over a large number of small rooms, with minimal required localization infrastructure. The idea is to accurately track the position of handheld devices or mobile robots, without interfering with its architecture. Using Ultra-Wide Band (UWB) devices, we leveraged our expertise in distributed and collaborative robotic systems to develop an novel solution requiring a minimal number of fixed anchors. We discuss a strategy to share the UWB network together with an Extended Kalman filter derivation to collaboratively locate and track UWB-equipped devices, and show results from our experimental campaign tracking visitors in the Chambord castle in France.

Ryan Cotsakis, David St-Onge, Giovanni Beltrame

Small unmanned aerial vehicles (UAVs) have generally little capacity to carry payloads. Through collaboration, the UAVs can increase their joint payload capacity and carry more significant loads. For maximum flexibility to dynamic and unstructured environments and task demands, we propose a fully decentralized control infrastructure based on a swarm-specific scripting language, Buzz. In this paper, we describe the control infrastructure and use it to compare two algorithms for collaborative transport: field potentials and spring-damper. We test the performance of our approach with a fleet of micro-UAVs, demonstrating the potential of decentralized control for collaborative transport.

David St-Onge, Vivek Shankar Varadharajan, Guannan Li et al.

This paper address the challenges encountered by developers when deploying a distributed decision-making behavior on heterogeneous robotic systems. Many applications benefit from the use of multiple robots, but their scalability and applicability are fundamentally limited if relying on a central control station. Getting beyond the centralized approach can increase the complexity of the embedded intelligence, the sensitivity to the network topology, and render the deployment on physical robots tedious and error-prone. By integrating the swarm-oriented programming language Buzz with the standard environment of ROS, this work demonstrates that behaviors requiring distributed consensus can be successfully deployed in practice. From simulation to the field, the behavioral script stays untouched and applicable to heterogeneous robot teams. We present the software structure of our solution as well as the swarm-oriented paradigms required from Buzz to implement a robust generic consensus strategy. We show the applicability of our solution with simulations and experiments with heterogeneous ground-and-air robotic teams.