The Differentiable Cross-Entropy Method
This work addresses the challenge of using CEM in machine learning pipelines, which was previously impossible, benefiting researchers and practitioners in optimization and control domains, though it appears incremental as it builds on existing CEM with a differentiable extension.
The authors tackled the problem of integrating the cross-entropy method (CEM) into end-to-end learning pipelines by introducing a differentiable variant, enabling differentiation with respect to objective function parameters, and demonstrated applications in synthetic structured prediction and non-convex continuous control, showing fine-tuning of CEM-based controllers with policy optimization.
We study the cross-entropy method (CEM) for the non-convex optimization of a continuous and parameterized objective function and introduce a differentiable variant that enables us to differentiate the output of CEM with respect to the objective function's parameters. In the machine learning setting this brings CEM inside of the end-to-end learning pipeline where this has otherwise been impossible. We show applications in a synthetic energy-based structured prediction task and in non-convex continuous control. In the control setting we show how to embed optimal action sequences into a lower-dimensional space. DCEM enables us to fine-tune CEM-based controllers with policy optimization.