Simple Policy Evaluation for Data-Rich Iterative Tasks
For roboticists working on iterative tasks, this method offers a computationally cheaper alternative to model-based control without sacrificing performance.
The paper presents a model-free policy evaluation method for iterative control tasks that uses safe trajectories to construct a piecewise affine value function approximation. It achieves a tenfold reduction in computation time while matching the performance of a model-based controller on a race car platform.
A data-based policy for iterative control task is presented. The proposed strategy is model-free and can be applied whenever safe input and state trajectories of a system performing an iterative task are available. These trajectories, together with a user-defined cost function, are exploited to construct a piecewise affine approximation to the value function. Approximated value functions are then used to evaluate the control policy by solving a linear program. We show that for linear system subject to convex cost and constraints, the proposed strategy guarantees closed-loop constraint satisfaction and performance bounds on the closed-loop trajectory. We evaluate the proposed strategy in simulations and experiments, the latter carried out on the Berkeley Autonomous Race Car (BARC) platform. We show that the proposed strategy is able to reduce the computation time by one order of magnitude while achieving the same performance as our model-based control algorithm.