Efficient search of active inference policy spaces using k-means
This work addresses a computational bottleneck in active inference for researchers and practitioners dealing with large state spaces, representing an incremental improvement.
The paper tackles the problem of intractable policy selection in active inference for large graphs by mapping policies to a vector space and using k-means clustering to efficiently search promising areas, achieving tractability for moderately large graphs.
We develop an approach to policy selection in active inference that allows us to efficiently search large policy spaces by mapping each policy to its embedding in a vector space. We sample the expected free energy of representative points in the space, then perform a more thorough policy search around the most promising point in this initial sample. We consider various approaches to creating the policy embedding space, and propose using k-means clustering to select representative points. We apply our technique to a goal-oriented graph-traversal problem, for which naive policy selection is intractable for even moderately large graphs.