OPEB: Open Physical Environment Benchmark for Artificial Intelligence
This addresses the need for standardized evaluation tools to facilitate research in AI for continuous-control tasks, though it is incremental as it builds on existing benchmark concepts.
The authors tackled the lack of standardized benchmarks for continuous-control AI tasks by proposing OPEB, a cloud-based physical environment benchmark framework that enables collaborative integration and sharing of benchmarks, demonstrated with a mountain-car example using reinforcement learning.
Artificial Intelligence methods to solve continuous- control tasks have made significant progress in recent years. However, these algorithms have important limitations and still need significant improvement to be used in industry and real- world applications. This means that this area is still in an active research phase. To involve a large number of research groups, standard benchmarks are needed to evaluate and compare proposed algorithms. In this paper, we propose a physical environment benchmark framework to facilitate collaborative research in this area by enabling different research groups to integrate their designed benchmarks in a unified cloud-based repository and also share their actual implemented benchmarks via the cloud. We demonstrate the proposed framework using an actual implementation of the classical mountain-car example and present the results obtained using a Reinforcement Learning algorithm.