Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition
This addresses the problem of training AI agents for fuzzy tasks without clear reward functions, primarily for researchers in reinforcement learning and human-in-the-loop AI, though it is incremental as it builds on existing competition frameworks.
The paper organized the MineRL BASALT 2022 competition to develop algorithms for solving tasks with hard-to-specify rewards in Minecraft using human feedback, resulting in top solutions that advanced methods for fine-tuning foundation models from such feedback.
To facilitate research in the direction of fine-tuning foundation models from human feedback, we held the MineRL BASALT Competition on Fine-Tuning from Human Feedback at NeurIPS 2022. The BASALT challenge asks teams to compete to develop algorithms to solve tasks with hard-to-specify reward functions in Minecraft. Through this competition, we aimed to promote the development of algorithms that use human feedback as channels to learn the desired behavior. We describe the competition and provide an overview of the top solutions. We conclude by discussing the impact of the competition and future directions for improvement.