Asymmetric Norms to Approximate the Minimum Action Distance
This provides a more accurate state representation for reward-free environments, particularly useful for goal-conditioned policies and planning in asymmetric settings.
The paper tackles the problem of approximating minimum action distances in Markov decision processes by learning an embedding space with an asymmetric norm parametrization, which performs comparably to symmetric norms in symmetric environments and surpasses them in asymmetric ones.
This paper presents a state representation for reward-free Markov decision processes. The idea is to learn, in a self-supervised manner, an embedding space where distances between pairs of embedded states correspond to the minimum number of actions needed to transition between them. Unlike previous methods, our approach incorporates an asymmetric norm parametrization, enabling accurate approximations of minimum action distances in environments with inherent asymmetry. We show how this representation can be leveraged to learn goal-conditioned policies, providing a notion of similarity between states and goals and a useful heuristic distance to guide planning. To validate our approach, we conduct empirical experiments on both symmetric and asymmetric environments. Our results show that our asymmetric norm parametrization performs comparably to symmetric norms in symmetric environments and surpasses symmetric norms in asymmetric environments.