Jnaneshwar Das

Harish Anand, Stephen A. Rees, Zhiang Chen et al.

Simulations play a crucial role in robotics research and education. This paper presents the OpenUAV testbed, an open-source, easy-to-use, web-based, and reproducible software system that enables students and researchers to run robotic simulations on the cloud. We have built upon our previous work and have addressed some of the educational and research challenges associated with the prior work. The critical contributions of the paper to the robotics and automation community are threefold: First, OpenUAV saves students and researchers from tedious and complicated software setups by providing web-browser-based Linux desktop sessions with standard robotics software like Gazebo, ROS, and flight autonomy stack. Second, a method for saving an individual's research work with its dependencies for the work's future reproducibility. Third, the platform provides a mechanism to support photorealistic robotics simulations by combining Unity game engine-based camera rendering and Gazebo physics. The paper addresses a research need for photorealistic simulations and describes a methodology for creating a photorealistic aquatic simulation. We also present the various academic and research use-cases of this platform to improve robotics education and research, especially during times like the COVID-19 pandemic, when virtual collaboration is necessary.

Zhiang Chen, J Ramon Arrowsmith, Jnaneshwar Das

Robotic mapping is useful in scientific applications that involve surveying unstructured environments. This paper presents a target-oriented mapping system for sparsely distributed geologic surface features, such as precariously balanced rocks (PBRs), whose geometric fragility parameters can provide valuable information on earthquake shaking history and landscape development for a region. With this geomorphology problem as the test domain, we demonstrate a pipeline for detecting, localizing, and precisely mapping fragile geologic features distributed on a landscape. To do so, we first carry out a lawn-mower search pattern in the survey region from a high elevation using an Unpiloted Aerial Vehicle (UAV). Once a potential PBR target is detected by a deep neural network, we track the bounding box in the image frames using a real-time tracking algorithm. The location and occupancy of the target in world coordinates are estimated using a sampling-based filtering algorithm, where a set of 3D points are re-sampled after weighting by the tracked bounding boxes from different camera perspectives. The converged 3D points provide a prior on 3D bounding shape of a target, which is used for UAV path planning to closely and completely map the target with Simultaneous Localization and Mapping (SLAM). After target mapping, the UAV resumes the lawn-mower search pattern to find the next target. We introduce techniques to make the target mapping robust to false positive and missing detection from the neural network. Our target-oriented mapping system has the advantages of reducing map storage and emphasizing complete visible surface features on specified targets.

Darwin Mick, K. Srikar Siddarth, Swastik Nandan et al.

Remote sample recovery is a rapidly evolving application of Small Unmanned Aircraft Systems (sUAS) for planetary sciences and space exploration. Development of cyber-physical systems (CPS) for autonomous deployment and recovery of sensor probes for sample caching is already in progress with NASA's MARS 2020 mission. To challenge student teams to develop autonomy for sample recovery settings, the 2020 NSF CPS Challenge was positioned around the launch of the MARS 2020 rover and sUAS duo. This paper discusses perception and trajectory planning for sample recovery by sUAS in a simulation environment. Out of a total of five teams that participated, the results of the top two teams have been discussed. The OpenUAV cloud simulation framework deployed on the Cyber-Physical Systems Virtual Organization (CPS-VO) allowed the teams to work remotely over a month during the COVID-19 pandemic to develop and simulate autonomous exploration algorithms. Remote simulation enabled teams across the globe to collaborate in experiments. The two teams approached the task of probe search, probe recovery, and landing on a moving target differently. This paper is a summary of teams' insights and lessons learned, as they chose from a wide range of perception sensors and algorithms.

Zhiang Chen, Sarah Bearman, J Ramon Arrowsmith et al.

Robotic mapping is attractive in many scientific applications that involve environmental surveys. This paper presents a system for localization and mapping of sparsely distributed surface features such as precariously balanced rocks (PBRs), whose geometric fragility parameters provide valuable information on earthquake processes and landscape development. With this geomorphologic problem as the test domain, we carry out a lawn-mower search pattern from a high elevation using an Unpiloted Aerial Vehicle (UAV) equipped with a flight controller, GPS module, stereo camera, and onboard computer. Once a potential PBR target is detected by a deep neural network in real time, we track its bounding box in the image coordinates by applying a Kalman filter that fuses the deep learning detection with Kanade-Lucas-Tomasi (KLT) tracking. The target is localized in world coordinates using depth filtering where a set of 3D points are filtered by object bounding boxes from different camera perspectives. The 3D points also provide a strong prior on target shape, which is used for UAV path planning to closely map the target using RGBD SLAM. After target mapping, the UAS resumes the lawn-mower search pattern to locate and map the next target.

Xu Liu, Steven W. Chen, Shreyas Aditya et al.

We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.