Difference of Convex Functions Programming Applied to Control with Expert Data
This work addresses performance enhancement in control algorithms using expert data, but it appears incremental as it applies an existing optimization technique to specific methods without claiming broad breakthroughs.
The paper tackles the problem of improving Learning from Demonstrations and Reinforcement Learning with expert data by applying Difference of Convex functions programming to optimize the Optimal Bellman Residual, demonstrating performance improvements on RCAL and RLED algorithms in experiments with generic MDPs.
This paper reports applications of Difference of Convex functions (DC) programming to Learning from Demonstrations (LfD) and Reinforcement Learning (RL) with expert data. This is made possible because the norm of the Optimal Bellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is DC. Improvement in performance is demonstrated on two specific algorithms, namely Reward-regularized Classification for Apprenticeship Learning (RCAL) and Reinforcement Learning with Expert Demonstrations (RLED), through experiments on generic Markov Decision Processes (MDP), called Garnets.