Nare Karapetyan

Akshaj Gaur, Cheng Liu, Xiaomin Lin et al.

With a number of marine populations in rapid decline, collecting and analyzing data about marine populations has become increasingly important to develop effective conservation policies for a wide range of marine animals, including whales. Modern computer vision algorithms allow us to detect whales in images in a wide range of domains, further speeding up and enhancing the monitoring process. However, these algorithms heavily rely on large training datasets, which are challenging and time-consuming to collect particularly in marine or aquatic environments. Recent advances in AI however have made it possible to synthetically create datasets for training machine learning algorithms, thus enabling new solutions that were not possible before. In this work, we present a solution - SeaDroneSim2 benchmark suite, which addresses this challenge by generating aerial, and satellite synthetic image datasets to improve the detection of whales and reduce the effort required for training data collection. We show that we can achieve a 15% performance boost on whale detection compared to using the real data alone for training, by augmenting a 10% real data. We open source both the code of the simulation platform SeaDroneSim2 and the dataset generated through it.

Xiaomin Lin, Nitin J. Sanket, Nare Karapetyan et al.

Oysters play a pivotal role in the bay living ecosystem and are considered the living filters for the ocean. In recent years, oyster reefs have undergone major devastation caused by commercial over-harvesting, requiring preservation to maintain ecological balance. The foundation of this preservation is to estimate the oyster density which requires accurate oyster detection. However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments. To this end, we present a novel method to mathematically model oysters and render images of oysters in simulation to boost the detection performance with minimal real data. Utilizing our synthetic data along with real data for oyster detection, we obtain up to 35.1% boost in performance as compared to using only real data with our OysterNet network. We also improve the state-of-the-art by 12.7%. This shows that using underlying geometrical properties of objects can help to enhance recognition task accuracy on limited datasets successfully and we hope more researchers adopt such a strategy for hard-to-obtain datasets.

Eric Chen, Travis Manderson, Nare Karapetyan et al.

Autonomous underwater vehicles (AUVs) are increasingly used to survey coral reefs, yet efficiently locating specific coral species of interest remains difficult: target species are often sparsely distributed across the reef, and an AUV with limited battery life cannot afford to search everywhere. When detections of the target itself are too sparse to provide directional guidance, the robot benefits from an additional signal to decide where to look next. We propose using the visual environmental context -- the habitat features that tend to co-occur with a target species -- as that signal. Because context features are spatially denser and often vary more smoothly than target detections, we hypothesize that a reward function targeted at broader environmental context will enable adaptive planners to make better decisions on where to go next, even in regions where no target has yet been observed. Starting from a single labeled image, our method uses patch-level DINOv2 embeddings to perform one-shot detections of both the target species and its surrounding context online. We validate our approach using real imagery collected by an AUV at two reef sites in St. John, U.S. Virgin Islands, simulating the robot's motion offline. Our results demonstrate that one-shot detection combined with adaptive context modeling enables efficient autonomous surveying, sampling up to 75$\%$ of the target in roughly half the time required by exhaustive coverage when the target is sparsely distributed, and outperforming search strategies that only use target detections.

Bharat Joshi, Sharmin Rahman, Michail Kalaitzakis et al.

A plethora of state estimation techniques have appeared in the last decade using visual data, and more recently with added inertial data. Datasets typically used for evaluation include indoor and urban environments, where supporting videos have shown impressive performance. However, such techniques have not been fully evaluated in challenging conditions, such as the marine domain. In this paper, we compare ten recent open-source packages to provide insights on their performance and guidelines on addressing current challenges. Specifically, we selected direct methods and tightly-coupled optimization techniques that fuse camera and Inertial Measurement Unit (IMU) data together. Experiments are conducted by testing all packages on datasets collected over the years with underwater robots in our laboratory. All the datasets are made available online.

Yisheng Zhang, Michael Xu, Alan Williams et al.

Accurate seabed mapping is essential for habitat monitoring and infrastructure inspection. In turbid, shallow coastal waters, such as shellfish aquaculture farms, the effectiveness of traditional optical methods is limited. Autonomous surface vehicles (ASVs) equipped with forward-looking sonar (FLS) offer a promising alternative. However, existing sonar-based systems face challenges in achieving fine resolution mapping over long trajectories due to low-resolution positioning measurements and accumulated drift over long trajectories. In this paper, we present a drift-resilient seabed mapping framework that integrates local FLS frame alignment using the Fourier-Mellin transform (FMT) with global trajectory optimization based on an extended Kalman filter (EKF) that fuses global positioning system (GPS), inertial measurement unit (IMU), and compass data. A variance-based image blending strategy is used to further reduce visual artifacts in overlapping regions. Field trials on a structured oyster farm site show that our framework helps reduce drift in RMSE by 9.5% relative to the FMT-only baseline. This framework also enables sub-meter reconstruction accuracy and preservation of high-resolution textures needed for oyster inventory estimation within the mapped areas.

Zhenqi Wu, Abhinav Modi, Angelos Mavrogiannis et al.

The ocean is warming and acidifying, increasing the risk of mass mortality events for temperature-sensitive shellfish such as oysters. This motivates the development of long-term monitoring systems. However, human labor is costly and long-duration underwater work is highly hazardous, thus favoring robotic solutions as a safer and more efficient option. To enable underwater robots to make real-time, environment-aware decisions without human intervention, we must equip them with an intelligent "brain." This highlights the need for persistent,wide-area, and low-cost benthic monitoring. To this end, we present DREAM, a Vision Language Model (VLM)-guided autonomy framework for long-term underwater exploration and habitat monitoring. The results show that our framework is highly efficient in finding and exploring target objects (e.g., oysters, shipwrecks) without prior location information. In the oyster-monitoring task, our framework takes 31.5% less time than the previous baseline with the same amount of oysters. Compared to the vanilla VLM, it uses 23% fewer steps while covering 8.88% more oysters. In shipwreck scenes, our framework successfully explores and maps the wreck without collisions, requiring 27.5% fewer steps than the vanilla model and achieving 100% coverage, while the vanilla model achieves 60.23% average coverage in our shipwreck environments.

Marios Xanthidis, Michail Kalaitzakis, Nare Karapetyan et al.

Visual monitoring operations underwater require both observing the objects of interest in close-proximity, and tracking the few feature-rich areas necessary for state estimation.This paper introduces the first navigation framework, called AquaVis, that produces on-line visibility-aware motion plans that enable Autonomous Underwater Vehicles (AUVs) to track multiple visual objectives with an arbitrary camera configuration in real-time. Using the proposed pipeline, AUVs can efficiently move in 3D, reach their goals while avoiding obstacles safely, and maximizing the visibility of multiple objectives along the path within a specified proximity. The method is sufficiently fast to be executed in real-time and is suitable for single or multiple camera configurations. Experimental results show the significant improvement on tracking multiple automatically-extracted points of interest, with low computational overhead and fast re-planning times

Nare Karapetyan, Jason Moulton, Ioannis Rekleitis

This paper addresses the problem of robotic operations in the presence of adversarial forces. We presents a complete framework for survey operations: waypoint generation,modelling of forces and tuning the control. In many applications of environmental monitoring, search and exploration, and bathymetric mapping, the vehicle has to traverse in straight lines parallel to each other, ensuring there are no gaps and no redundant coverage. During operations with an Autonomous Surface Vehicle (ASV) however, the presence of wind and/or currents produces external forces acting on the vehicle which quite often divert it from its intended path. Similar issues have been encountered during aerial or underwater operations. By measuring these phenomena, wind and current, and modelling their impact on the vessel, actions can be taken to alleviate their effect and ensure the correct trajectory is followed.

Jason Moulton, Nare Karapetyan, Michail Kalaitzakis et al.

The ability to navigate, search, and monitor dynamic marine environments such as ports, deltas, tributaries, and rivers presents several challenges to both human operated and autonomously operated surface vehicles. Human data collection and monitoring is overly taxing and inconsistent when faced with large coverage areas, disturbed environments, and potentially uninhabitable situations. In contrast,the same missions become achievable with Autonomous Surface Vehicles (ASVs)configured and capable of accurately maneuvering in such environments. The two dynamic factors that present formidable challenges to completing precise maneuvers in coastal and moving waters are currents and winds. In this work, we present novel and inexpensive methods for sensing these external forces, together with methods for accurately controlling an ASV in the presence of such external forces. The resulting platform is capable of deploying bathymetric and water quality monitoring sensors. Experimental results in local lakes and rivers demonstrate the feasibility of the proposed approach.

Nare Karapetyan, Adam Braude, Jason Moulton et al.

Environmental monitoring and surveying operations on rivers currently are performed primarily with manually-operated boats. In this domain, autonomous coverage of areas is of vital importance, for improving both the quality and the efficiency of coverage. This paper leverages human expertise in river exploration and data collection strategies to automate and optimize these processes using autonomous surface vehicles(ASVs). In particular, three deterministic algorithms for both partial and complete coverage of a river segment are proposed,providing varying path length, coverage density, and turning patterns. These strategies resulted in increases in accuracy and efficiency compared to human performance.The proposed methods were extensively tested in simulation using maps of real rivers of different shapes and sizes. In addition, to verify their performance in real world operations, the algorithms were deployed successfully on several parts of the Congaree River in South Carolina, USA, resulting in total of more than 35km of coverage trajectories in the field.

Nare Karapetyan, Jason Moulton, Ioannis Rekleitis

Autonomous coverage has tremendous importance for environmental surveying and exploration tasks performed on rivers both in terms of efficiency and data collection quality. Most surveys of rivers are performed manually using quite similar approaches. Using these practices to automate these processes improves the quality of survey operations. In addition to human expertise on the type of patterns,the coverage of a river can be optimized using the river meanders to determine the direction of coverage. In this work we use the implicit information on the speed of the water current, inferred from the curves of the river, to reduce the cost of cover-age. We use autonomous surface vehicles (ASVs) to deploy the proposed methods and demonstrate the efficiency of our method. In addition we compare the proposed method with previous coverage techniques developed in our lab. When taking into account meanders the coverage time has been decreased in average by more than 20%. The deployments of the ASVs were performed on the Congaree River, SC,USA, and resulted in more than 27km of total coverage trajectories.

Marios Xanthidis, Nare Karapetyan, Hunter Damron et al.

Navigation underwater traditionally is done by keeping a safe distance from obstacles, resulting in "fly-overs" of the area of interest. Movement of an autonomous underwater vehicle (AUV) through a cluttered space, such as a shipwreck or a decorated cave, is an extremely challenging problem that has not been addressed in the past. This paper proposes a novel navigation framework utilizing an enhanced version of Trajopt for fast 3D path-optimization planning for AUVs. A sampling-based correction procedure ensures that the planning is not constrained by local minima, enabling navigation through narrow spaces. Two different modalities are proposed: planning with a known map results in efficient trajectories through cluttered spaces; operating in an unknown environment utilizes the point cloud from the visual features detected to navigate efficiently while avoiding the detected obstacles. The proposed approach is rigorously tested, both on simulation and in-pool experiments, proven to be fast enough to enable safe real-time 3D autonomous navigation for an AUV.

Jason Moulton, Nare Karapetyan, Alberto Quattrini Li et al.

Operating in the presence of strong adverse forces is a particularly challenging problem in field robotics. In most robotic operations where the robot is not firmly grounded, such as aerial, surface, and underwater, minimal external forces are assumed as the standard operating procedures. The first action for operating in the presence of non-trivial forces is modeling the forces and their effect on the robots motion. In this work an Autonomous Surface Vehicle (ASV), operating on lakes and rivers with varying winds and currents, collects wind and current measurements with an inexpensive custom-made sensor suite setup, and generates a model of the force field. The modeling process takes into account depth, wind, and current measurements along with the ASVs trajectory from GPS. In this work, we propose a method for an ASV to build an environmental force map by integrating in a Gaussian Process the wind, depth, and current measurements gathered at the surface. We run extensive experimental field trials for our approach on real Jetyak ASVs. Experimental results from different locations validate the proposed modeling approach.

Jason Moulton, Nare Karapetyan, Sharon Bukhsbaum et al.

Environmental monitoring of marine environments presents several challenges: the harshness of the environment, the often remote location, and most importantly, the vast area it covers. Manual operations are time consuming, often dangerous, and labor intensive. Operations from oceanographic vessels are costly and limited to open seas and generally deeper bodies of water. In addition, with lake, river, and ocean shoreline being a finite resource, waterfront property presents an ever increasing valued commodity, requiring exploration and continued monitoring of remote waterways. In order to efficiently explore and monitor currently known marine environments as well as reach and explore remote areas of interest, we present a design of an autonomous surface vehicle (ASV) with the power to cover large areas, the payload capacity to carry sufficient power and sensor equipment, and enough fuel to remain on task for extended periods. An analysis of the design and a discussion on lessons learned during deployments is presented in this paper.

Nare Karapetyan, Jason Moulton, Jeremy S. Lewis et al.

In large scale coverage operations, such as marine exploration or aerial monitoring, single robot approaches are not ideal, as they may take too long to cover a large area. In such scenarios, multi-robot approaches are preferable. Furthermore, several real world vehicles are non-holonomic, but can be modeled using Dubins vehicle kinematics. This paper focuses on environmental monitoring of aquatic environments using Autonomous Surface Vehicles (ASVs). In particular, we propose a novel approach for solving the problem of complete coverage of a known environment by a multi-robot team consisting of Dubins vehicles. It is worth noting that both multi-robot coverage and Dubins vehicle coverage are NP-complete problems. As such, we present two heuristics methods based on a variant of the traveling salesman problem -- k-TSP -- formulation and clustering algorithms that efficiently solve the problem. The proposed methods are tested both in simulations to assess their scalability and with a team of ASVs operating on a lake to ensure their applicability in real world.

Nare Karapetyan, Kelly Benson, Chris McKinney et al.

This paper addresses the complete area coverage problem of a known environment by multiple-robots. Complete area coverage is the problem of moving an end-effector over all available space while avoiding existing obstacles. In such tasks, using multiple robots can increase the efficiency of the area coverage in terms of minimizing the operational time and increase the robustness in the face of robot attrition. Unfortunately, the problem of finding an optimal solution for such an area coverage problem with multiple robots is known to be NP-complete. In this paper we present two approximation heuristics for solving the multi-robot coverage problem. The first solution presented is a direct extension of an efficient single robot area coverage algorithm, based on an exact cellular decomposition. The second algorithm is a greedy approach that divides the area into equal regions and applies an efficient single-robot coverage algorithm to each region. We present experimental results for two algorithms. Results indicate that our approaches provide good coverage distribution between robots and minimize the workload per robot, meanwhile ensuring complete coverage of the area.