Achieving Fairness in DareFightingICE Agents Evaluation Through a Delay Mechanism
This addresses fairness issues in agent evaluation for a specific fighting game research platform, with potential broader implications for gRPC-based applications.
This paper tackled the problem of unfair agent performance comparisons in DareFightingICE due to gRPC latency differences between Java and Python, finding that a proposed delay mechanism equalized performance, making Java-based and Python-based agents perform similarly.
This paper proposes a delay mechanism to mitigate the impact of latency differences in the gRPC framework--a high-performance, open-source universal remote procedure call (RPC) framework--between different programming languages on the performance of agents in DareFightingICE, a fighting game research platform. The study finds that gRPC latency differences between Java and Python can significantly impact real-time decision-making. Without a delay mechanism, Java-based agents outperform Python-based ones due to lower gRPC latency on the Java platform. However, with the proposed delay mechanism, both Java-based and Python-based agents exhibit similar performance, leading to a fair comparison between agents developed using different programming languages. Thus, this work underscores the crucial importance of considering gRPC latency when developing and evaluating agents in DareFightingICE, and the insights gained could potentially extend to other gRPC-based applications.