Improving Language Model Prompting in Support of Semi-autonomous Task Learning
This work addresses the challenge of enabling semi-autonomous agents to learn tasks more effectively by improving LLM prompting, though it appears incremental as it builds on existing knowledge extraction methods.
The paper tackled the problem of generating effective prompts for language models to provide actionable knowledge for agents learning new tasks, demonstrating that actionable task knowledge can be obtained from LLMs to support online agent task learning.
Language models (LLMs) offer potential as a source of knowledge for agents that need to acquire new task competencies within a performance environment. We describe efforts toward a novel agent capability that can construct cues (or "prompts") that result in useful LLM responses for an agent learning a new task. Importantly, responses must not only be "reasonable" (a measure used commonly in research on knowledge extraction from LLMs) but also specific to the agent's task context and in a form that the agent can interpret given its native language capacities. We summarize a series of empirical investigations of prompting strategies and evaluate responses against the goals of targeted and actionable responses for task learning. Our results demonstrate that actionable task knowledge can be obtained from LLMs in support of online agent task learning.