Nonsmooth optimal value and policy functions in mechanical systems subject to unilateral constraints
This identifies a fundamental limitation for applying smooth approximation methods in contact-rich robotics, which is incremental as it highlights an existing issue rather than proposing a new solution.
The paper demonstrates that value and policy functions in optimal control for mechanical systems with unilateral constraints, such as robot locomotion and manipulation, are generally nonsmooth due to discontinuous or piecewise-differentiable dynamics, using simple examples to illustrate this finding.
State-of-the-art approaches to optimal control use smooth approximations of value and policy functions and gradient-based algorithms for improving approximator parameters. Unfortunately, we show that value and policy functions that arise in optimal control of mechanical systems subject to unilateral constraints -- i.e. the contact-rich dynamics of robot locomotion and manipulation -- are generally nonsmooth due to the underlying dynamics exhibiting discontinuous or piecewise-differentiable trajectory outcomes. Simple mechanical systems are used to illustrate this result and the implications for optimal control of contact-rich robot dynamics.