Tracking control of latent dynamic systems with application to spacecraft attitude control
This work addresses sample inefficiency and generalization issues in reinforcement learning for latent dynamic systems, with application to spacecraft attitude control, but it appears incremental as it builds on existing control methods.
The authors tackled the problem of controlling latent dynamic systems, where controllable variables are inferred from high-dimensional observations, by proposing an asymptotic tracking controller that learns an identifiable latent dynamic model and uses feedback linearization, achieving asymptotic tracking in simulations including noise and control deviations.
When intelligent spacecraft or space robots perform tasks in a complex environment, the controllable variables are usually not directly available and have to be inferred from high-dimensional observable variables, such as outputs of neural networks or images. While the dynamics of these observations are highly complex, the mechanisms behind them may be simple, which makes it possible to regard them as latent dynamic systems. For control of latent dynamic systems, methods based on reinforcement learning suffer from sample inefficiency and generalization problems. In this work, we propose an asymptotic tracking controller for latent dynamic systems. The latent variables are related to the high-dimensional observations through an unknown nonlinear function. The dynamics are unknown but assumed to be affine nonlinear. To realize asymptotic tracking, an identifiable latent dynamic model is learned to recover the latents and estimate the dynamics. This training process does not depend on the goals or reference trajectories. Based on the learned model, we use a manually designed feedback linearization controller to ensure the asymptotic tracking property of the closed-loop system. After considering fully controllable systems, the results are extended to the case that uncontrollable environmental latents exist. As an application, simulation experiments on a latent spacecraft attitude dynamic model are conducted to verify the proposed methods, and the observation noise and control deviation are taken into consideration.