Automated Generation of Massive Reasonable Empirical Theorems by Forward Reasoning Based on Strong Relevant Logics -- A Solution to the Problem of LLM Pre-training Data Exhaustion
This addresses the data scarcity issue for LLM developers, but appears incremental as it builds on existing automated theorem finding and knowledge appreciation methods.
The paper tackles the problem of data exhaustion for large language model pre-training by proposing an automated method to generate massive reasonable empirical theorems using forward reasoning based on strong relevant logics, which is part of their approach to automated theorem finding and knowledge appreciation.
Recently, it is often said that the data used for the pre-training of large language models (LLMs) have been exhausted. This paper proposes a solution to the problem: Automated generation of massive reasonable empirical theorems by forward reasoning based on strong relevant logics. In fact, this can be regarded as a part of our approach to the problems of ATF (Automated Theorem Finding) and AKA (Automated Knowledge Appreciation).