Differentiable Weightless Controllers: Learning Logic Circuits for Continuous Control
This addresses the need for low-latency, energy-efficient control systems, such as in robotics or embedded applications, by enabling FPGA-compatible circuits with nanojoule-level energy costs, though it is incremental in applying symbolic-differentiable methods to control.
The paper tackles the problem of representing continuous-control policies as discrete logic circuits instead of neural networks, introducing Differentiable Weightless Controllers (DWCs) that achieve returns competitive with weight-based policies across five MuJoCo benchmarks, matching performance on four tasks.
We investigate whether continuous-control policies can be represented and learned as discrete logic circuits instead of continuous neural networks. We introduce Differentiable Weightless Controllers (DWCs), a symbolic-differentiable architecture that maps real-valued observations to actions using thermometer-encoded inputs, sparsely connected boolean lookup-table layers, and lightweight action heads. DWCs can be trained end-to-end by gradient-based techniques, yet compile directly into FPGA-compatible circuits with few- or even single-clock-cycle latency and nanojoule-level energy cost per action. Across five MuJoCo benchmarks, including high-dimensional Humanoid, DWCs achieve returns competitive with weight-based policies (full precision or quantized neural networks), matching performance on four tasks and isolating network capacity as the key limiting factor on HalfCheetah. Furthermore, DWCs exhibit structurally sparse and interpretable connectivity patterns, enabling a direct inspection of which input thresholds influence control decisions.