Language Models are Crossword Solvers
This addresses the problem of automated crossword solving for AI researchers, showing incremental improvements in a domain-specific task.
The paper tackled solving crosswords with large language models, demonstrating that they outperform previous state-of-the-art results by a factor of 2-3 in benchmarks and achieve 93% accuracy on New York Times puzzles.
Crosswords are a form of word puzzle that require a solver to demonstrate a high degree of proficiency in natural language understanding, wordplay, reasoning, and world knowledge, along with adherence to character and length constraints. In this paper we tackle the challenge of solving crosswords with large language models (LLMs). We demonstrate that the current generation of language models shows significant competence at deciphering cryptic crossword clues and outperforms previously reported state-of-the-art (SoTA) results by a factor of 2-3 in relevant benchmarks. We also develop a search algorithm that builds off this performance to tackle the problem of solving full crossword grids with out-of-the-box LLMs for the very first time, achieving an accuracy of 93% on New York Times crossword puzzles. Additionally, we demonstrate that LLMs generalize well and are capable of supporting answers with sound rationale.