State Estimation and Control of Dynamic Systems from High-Dimensional Image Data
This work addresses the challenge of policy learning in dynamic systems without direct state access, though it appears incremental as it integrates existing methods like CNNs and GRUs for a specific application.
The paper tackled the problem of accurate state estimation from high-dimensional image data in dynamic systems, where obtaining true states is impractical, and introduced a neural architecture combining CNNs and GRUs to learn state representations for reinforcement learning, achieving real-time, accurate estimation and control without ground-truth states.
Accurate state estimation is critical for optimal policy design in dynamic systems. However, obtaining true system states is often impractical or infeasible, complicating the policy learning process. This paper introduces a novel neural architecture that integrates spatial feature extraction using convolutional neural networks (CNNs) and temporal modeling through gated recurrent units (GRUs), enabling effective state representation from sequences of images and corresponding actions. These learned state representations are used to train a reinforcement learning agent with a Deep Q-Network (DQN). Experimental results demonstrate that our proposed approach enables real-time, accurate estimation and control without direct access to ground-truth states. Additionally, we provide a quantitative evaluation methodology for assessing the accuracy of the learned states, highlighting their impact on policy performance and control stability.