Dataset and Performance Comparison of Deep Learning Architectures for Plum Detection and Robotic Harvesting
This work addresses labor shortages in agriculture by improving robotic harvesting, but it is incremental as it focuses on dataset creation and benchmarking existing methods.
The study tackled the problem of robust object detection for robotic plum harvesting under varying lighting and environmental conditions by creating new day and night datasets and benchmarking deep learning detectors, finding significant accuracy differences between day and night and that transfer learning is essential while depth fusion is only marginally effective.
Many automated operations in agriculture, such as weeding and plant counting, require robust and accurate object detectors. Robotic fruit harvesting is one of these, and is an important technology to address the increasing labour shortages and uncertainty suffered by tree crop growers. An eye-in-hand sensing setup is commonly used in harvesting systems and provides benefits to sensing accuracy and flexibility. However, as the hand and camera move from viewing the entire trellis to picking a specific fruit, large changes in lighting, colour, obscuration and exposure occur. Object detection algorithms used in harvesting should be robust to these challenges, but few datasets for assessing this currently exist. In this work, two new datasets are gathered during day and night operation of an actual robotic plum harvesting system. A range of current generation deep learning object detectors are benchmarked against these. Additionally, two methods for fusing depth and image information are tested for their impact on detector performance. Significant differences between day and night accuracy of different detectors is found, transfer learning is identified as essential in all cases, and depth information fusion is assessed as only marginally effective. The dataset and benchmark models are made available online.