Guided scenarios with simulated expert personae: a remarkable strategy to perform cognitive work
This approach offers a novel strategy for performing cognitive work using LLMs, potentially benefiting researchers and practitioners in AI and related fields, though it appears incremental in its application of existing LLM capabilities.
The paper tackles the problem of eliciting expert behavior from large language models (LLMs) by forming teams of simulated personae with guided scenarios, demonstrating its power through examples that address LLM factuality and reproduce a recent quantum optics result.
Large language models (LLMs) trained on a substantial corpus of human knowledge and literature productively work with a large array of facts from that corpus. Surprisingly, they are also able to re-create the behaviors of personae that are captured within the corpus. By forming teams of simulated personae, supplying contexts that set the stage, and providing gentle prompts, one can move through scenarios that elicit expert behavior to perform meaningful cognitive work. The power of this strategy is demonstrated with two examples, one attacking factuality of LLM responses and the other reproducing a very recently published result in quantum optics.