A Review of the Deep Sea Treasure problem as a Multi-Objective Reinforcement Learning Benchmark
This work addresses the need for more realistic benchmarks in multi-objective reinforcement learning, though it is incremental as it modifies an existing problem rather than creating a new one.
The authors analyzed the Deep Sea Treasure problem as a multi-objective reinforcement learning benchmark, finding it too simplistic and unrepresentative of practical problems, and proposed an improved version with proofs showing it overcomes these limitations while providing a reference implementation and complete Pareto-front.
In this paper, the authors investigate the Deep Sea Treasure (DST) problem as proposed by Vamplew et al. Through a number of proofs, the authors show the original DST problem to be quite basic, and not always representative of practical Multi-Objective Optimization problems. In an attempt to bring theory closer to practice, the authors propose an alternative, improved version of the DST problem, and prove that some of the properties that simplify the original DST problem no longer hold. The authors also provide a reference implementation and perform a comparison between their implementation, and other existing open-source implementations of the problem. Finally, the authors also provide a complete Pareto-front for their new DST problem.