Retrieval-Augmented Code Generation for Situated Action Generation: A Case Study on Minecraft
This work addresses action generation in collaborative AI tasks, but it is incremental as it applies existing LLM techniques to a specific domain.
The paper tackled the problem of predicting Builder action sequences in the Minecraft Collaborative Building Task using large language models (LLMs) with few-shot prompting, resulting in significant performance improvements over baseline methods.
In the Minecraft Collaborative Building Task, two players collaborate: an Architect (A) provides instructions to a Builder (B) to assemble a specified structure using 3D blocks. In this work, we investigate the use of large language models (LLMs) to predict the sequence of actions taken by the Builder. Leveraging LLMs' in-context learning abilities, we use few-shot prompting techniques, that significantly improve performance over baseline methods. Additionally, we present a detailed analysis of the gaps in performance for future work