Behavioral Cloning via Search in Embedded Demonstration Dataset
This work addresses policy adaptation in behavioral cloning for AI agents in complex environments like Minecraft, offering an incremental improvement with zero-shot capabilities.
The paper tackled the problem of behavioral cloning by proposing a search-based method over a latent space of expert demonstrations, enabling zero-shot task adaptation and achieving performance comparable to trained models on the BASALT MineRL-dataset in Minecraft.
Behavioural cloning uses a dataset of demonstrations to learn a behavioural policy. To overcome various learning and policy adaptation problems, we propose to use latent space to index a demonstration dataset, instantly access similar relevant experiences, and copy behavior from these situations. Actions from a selected similar situation can be performed by the agent until representations of the agent's current situation and the selected experience diverge in the latent space. Thus, we formulate our control problem as a search problem over a dataset of experts' demonstrations. We test our approach on BASALT MineRL-dataset in the latent representation of a Video PreTraining model. We compare our model to state-of-the-art Minecraft agents. Our approach can effectively recover meaningful demonstrations and show human-like behavior of an agent in the Minecraft environment in a wide variety of scenarios. Experimental results reveal that performance of our search-based approach is comparable to trained models, while allowing zero-shot task adaptation by changing the demonstration examples.