Complete Chess Games Enable LLM Become A Chess Master
This addresses the underexplored ability of LLMs to play abstract games like chess, which is incremental as it applies existing fine-tuning methods to a new domain.
The authors tackled the problem of enabling large language models to play chess by transforming the game into a textual format and using supervised fine-tuning, achieving a professional-level Elo rating of 1788 against Stockfish with 10 samples and showing that data quality improves performance by 350 Elo points.
Large language models (LLM) have shown remarkable abilities in text generation, question answering, language translation, reasoning and many other tasks. It continues to advance rapidly and is becoming increasingly influential in various fields, from technology and business to education and entertainment. Despite LLM's success in multiple areas, its ability to play abstract games, such as chess, is underexplored. Chess-playing requires the language models to output legal and reasonable moves from textual inputs. Here, we propose the Large language model ChessLLM to play full chess games. We transform the game into a textual format with the best move represented in the Forsyth-Edwards Notation. We show that by simply supervised fine-tuning, our model has achieved a professional-level Elo rating of 1788 in matches against the standard Elo-rated Stockfish when permitted to sample 10 times. We further show that data quality is important. Long-round data supervision enjoys a 350 Elo rating improvement over short-round data.