A Case Study on the Effectiveness of LLMs in Verification with Proof Assistants
This is an incremental study for researchers in formal verification and AI, assessing LLMs' utility in automating proof generation.
The paper tackled the problem of evaluating the effectiveness of large language models (LLMs) in generating proofs for verification with proof assistants, finding that LLMs perform well on small proofs but vary across projects and can make odd mistakes.
Large language models (LLMs) can potentially help with verification using proof assistants by automating proofs. However, it is unclear how effective LLMs are in this task. In this paper, we perform a case study based on two mature Rocq projects: the hs-to-coq tool and Verdi. We evaluate the effectiveness of LLMs in generating proofs by both quantitative and qualitative analysis. Our study finds that: (1) external dependencies and context in the same source file can significantly help proof generation; (2) LLMs perform great on small proofs but can also generate large proofs; (3) LLMs perform differently on different verification projects; and (4) LLMs can generate concise and smart proofs, apply classical techniques to new definitions, but can also make odd mistakes.