A Critique of Strictly Batch Imitation Learning
This is an incremental critique for researchers in imitation learning, highlighting potential flaws in a specific method.
The paper critiques a recent approach to offline imitation learning that uses joint energy-based models, arguing that it suffers from notational issues and can lead to inconsistent policy estimates compared to behavioral cloning.
Recent work by Jarrett et al. attempts to frame the problem of offline imitation learning (IL) as one of learning a joint energy-based model, with the hope of out-performing standard behavioral cloning. We suggest that notational issues obscure how the psuedo-state visitation distribution the authors propose to optimize might be disconnected from the policy's $\textit{true}$ state visitation distribution. We further construct natural examples where the parameter coupling advocated by Jarrett et al. leads to inconsistent estimates of the expert's policy, unlike behavioral cloning.