Learning Computational Efficient Bots with Costly Features
This work addresses computational bottlenecks for real-time AI applications, such as video games, by introducing a method to reduce inference costs while maintaining performance, representing an incremental improvement over existing approaches.
The paper tackles the trade-off between computational efficiency and task performance in deep reinforcement learning for real-time settings like video games, proposing a Budgeted Decision Transformer that dynamically selects input features to achieve similar performance with significantly fewer computational resources.
Deep reinforcement learning (DRL) techniques have become increasingly used in various fields for decision-making processes. However, a challenge that often arises is the trade-off between both the computational efficiency of the decision-making process and the ability of the learned agent to solve a particular task. This is particularly critical in real-time settings such as video games where the agent needs to take relevant decisions at a very high frequency, with a very limited inference time. In this work, we propose a generic offline learning approach where the computation cost of the input features is taken into account. We derive the Budgeted Decision Transformer as an extension of the Decision Transformer that incorporates cost constraints to limit its cost at inference. As a result, the model can dynamically choose the best input features at each timestep. We demonstrate the effectiveness of our method on several tasks, including D4RL benchmarks and complex 3D environments similar to those found in video games, and show that it can achieve similar performance while using significantly fewer computational resources compared to classical approaches.