Discretizing Dynamics for Maximum Likelihood Constraint Inference
This work addresses the challenge of inferring constraints from demonstrations in continuous systems, which is incremental as it adapts an existing technique to a more useful setting for applications like human-robot interaction and biomechanical medicine.
The paper tackled the problem of extending maximum likelihood constraint inference to continuous dynamics by discretizing them into a tabular state-action space, enabling accurate constraint inference on nonlinear pendulum systems with 2- and 4-dimensional state spaces and showing robustness to hyperparameters.
Maximum likelihood constraint inference is a powerful technique for identifying unmodeled constraints that affect the behavior of a demonstrator acting under a known objective function. However, it was originally formulated only for discrete state-action spaces. Continuous dynamics are more useful for modeling many real-world systems of interest, including the movements of humans and robots. We present a method to generate a tabular state-action space that approximates continuous dynamics and can be used for constraint inference on demonstrations that obey the true system dynamics. We then demonstrate accurate constraint inference on nonlinear pendulum systems with 2- and 4-dimensional state spaces, and show that performance is robust to a range of hyperparameters. The demonstrations are not required to be fully optimal with respect to the objective, and the most likely constraints can be identified even when demonstrations cover only a small portion of the state space. For these reasons, the proposed approach may be especially useful for inferring constraints on human demonstrators, which has important applications in human-robot interaction and biomechanical medicine.