Can Large Language Models Create New Knowledge for Spatial Reasoning Tasks?
This addresses the challenge of verifying LLM-generated novelty for researchers and developers, though it is incremental in demonstrating capabilities on specific tasks.
The paper investigates whether large language models (LLMs) can generate new knowledge for spatial reasoning tasks, finding that Claude 3 performs well on problems unlikely to have been directly encountered during training, indicating emergent properties.
The potential for Large Language Models (LLMs) to generate new information offers a potential step change for research and innovation. This is challenging to assert as it can be difficult to determine what an LLM has previously seen during training, making "newness" difficult to substantiate. In this paper we observe that LLMs are able to perform sophisticated reasoning on problems with a spatial dimension, that they are unlikely to have previously directly encountered. While not perfect, this points to a significant level of understanding that state-of-the-art LLMs can now achieve, supporting the proposition that LLMs are able to yield significant emergent properties. In particular, Claude 3 is found to perform well in this regard.