Stochastic Trajectory Influence Functions for LQR: Joint Sensitivity Through Dynamics and Noise Covariance
This work provides a method for data attribution in model-based control, which is incremental as it extends influence functions to stochastic LQR with joint sensitivity analysis.
The paper tackles the problem of attributing closed-loop performance in stochastic LQR to specific training trajectories, addressing biases from both learned dynamics and noise covariance. It develops a three-level influence hierarchy to approximate these effects without costly retraining.
Model-based controllers learned from data have the biases and noise of their training trajectories, making it important to know which trajectories help or hurt closed-loop performance. Influence functions, widely used in machine learning for data attribution, approximate this effect through first-order parameter-shift surrogates, avoiding costly retraining. Applying them to stochastic LQR, however, is nontrivial because the cost depends on the learned dynamics through the Riccati equation, and the process-noise covariance is estimated from the same residuals. We develop a three-level influence hierarchy that accounts for both channels.