A Cantor-Kantorovich Metric Between Markov Decision Processes with Application to Transfer Learning
This work addresses the challenge of predicting transfer learning outcomes in reinforcement learning, though it appears incremental as it builds directly on prior work for Markov chains.
The authors tackled the problem of measuring distances between Markov Decision Processes (MDPs) by extending the Cantor-Kantorovich metric from Markov chains to MDPs, showing it can be efficiently approximated and applied to forecast transfer learning algorithm performance with numerical evidence.
We extend the notion of Cantor-Kantorovich distance between Markov chains introduced by (Banse et al., 2023) in the context of Markov Decision Processes (MDPs). The proposed metric is well-defined and can be efficiently approximated given a finite horizon. Then, we provide numerical evidences that the latter metric can lead to interesting applications in the field of reinforcement learning. In particular, we show that it could be used for forecasting the performance of transfer learning algorithms.