A note on the empirical comparison of RBG and Ludii
This work addresses performance evaluation for AI researchers in general game playing, but it is incremental as it updates a prior flawed comparison.
The paper compares the efficiency of three General Game Playing systems (RBG, Ludii, and GDL propnet), finding that RBG is the fastest, with RBG being about 37 times faster than Ludii for chess, and Ludii being about 3 times slower than GDL propnet.
We present an experimental comparison of the efficiency of three General Game Playing systems in their current versions: Regular Boardgames (RBG 1.0), Ludii~0.3.0, and a Game Description Language (GDL) propnet. We show that in general, RBG is currently the fastest GGP system. For example, for chess, we demonstrate that RBG is about 37 times faster than Ludii, and Ludii is about 3 times slower than a GDL propnet. Referring to the recent comparison [An Empirical Evaluation of Two General Game Systems: Ludii and RBG, CoG 2019], we show evidences that the benchmark presented there contains a number of significant flaws that lead to wrong conclusions.