Research on Autonomous Maneuvering Decision of UCAV based on Approximate Dynamic Programming
This addresses the need for real-time optimal strategies in complex air combat scenarios for military or autonomous systems, but it appears incremental as it applies known reinforcement learning techniques to a specific domain.
The paper tackled the problem of autonomous maneuvering decision-making for unmanned combat aerial vehicles (UCAVs) in 3-D air combat by using an approximate dynamic programming approach, resulting in a method that provides fast responses and learns long-term strategies without explicit rule coding.
Unmanned aircraft systems can perform some more dangerous and difficult missions than manned aircraft systems. In some highly complicated and changeable tasks, such as air combat, the maneuvering decision mechanism is required to sense the combat situation accurately and make the optimal strategy in real-time. This paper presents a formulation of a 3-D one-on-one air combat maneuvering problem and an approximate dynamic programming approach for computing an optimal policy on autonomous maneuvering decision making. The aircraft learns combat strategies in a Reinforcement Leaning method, while sensing the environment, taking available maneuvering actions and getting feedback reward signals. To solve the problem of dimensional explosion in the air combat, the proposed method is implemented through feature selection, trajectory sampling, function approximation and Bellman backup operation in the air combat simulation environment. This approximate dynamic programming approach provides a fast response to a rapidly changing tactical situation, learns in long-term planning, without any explicitly coded air combat rule base.