Reinforcement Learning Control of a Forestry Crane Manipulator
This work addresses automation for forestry machines, offering incremental improvements in energy efficiency and success rates for log grasping tasks.
The study tackled the challenge of automating forestry crane manipulators in unstructured environments by applying reinforcement learning control, achieving a 97% grasping success rate and significantly reducing energy consumption with minimal increase in cycle time.
Forestry machines are heavy vehicles performing complex manipulation tasks in unstructured production forest environments. Together with the complex dynamics of the on-board hydraulically actuated cranes, the rough forest terrains have posed a particular challenge in forestry automation. In this study, the feasibility of applying reinforcement learning control to forestry crane manipulators is investigated in a simulated environment. Our results show that it is possible to learn successful actuator-space control policies for energy efficient log grasping by invoking a simple curriculum in a deep reinforcement learning setup. Given the pose of the selected logs, our best control policy reaches a grasping success rate of 97%. Including an energy-optimization goal in the reward function, the energy consumption is significantly reduced compared to control policies learned without incentive for energy optimization, while the increase in cycle time is marginal. The energy-optimization effects can be observed in the overall smoother motion and acceleration profiles during crane manipulation.