Reinforcement Learning for Robotic Time-optimal Path Tracking Using Prior Knowledge
This work addresses a domain-specific problem for industrial robotics by providing an incremental improvement in handling motor characteristics for path tracking.
The paper tackled time-optimal path tracking for industrial robots by addressing velocity-dependent actuator torque constraints, proposing an improved Q-learning algorithm that uses prior knowledge and reward-penalty mechanisms to achieve faster convergence and generate feasible trajectories.
Time-optimal path tracking, as a significant tool for industrial robots, has attracted the attention of numerous researchers. In most time-optimal path tracking problems, the actuator torque constraints are assumed to be conservative, which ignores the motor characteristic; i.e., the actuator torque constraints are velocity-dependent, and the relationship between torque and velocity is piecewise linear. However, considering that the motor characteristics increase the solving difficulty, in this study, an improved Q-learning algorithm for robotic time-optimal path tracking using prior knowledge is proposed. After considering the limitations of the Q-learning algorithm, an improved action-value function is proposed to improve the convergence rate. The proposed algorithms use the idea of reward and penalty, rewarding the actions that satisfy constraint conditions and penalizing the actions that break constraint conditions, to finally obtain a time-optimal trajectory that satisfies the constraint conditions. The effectiveness of the algorithms is verified by experiments.