Human-Centered Planning
This addresses the challenge of creating AI assistants that can generate flexible, human-executable plans based on natural language inputs, representing an incremental improvement over existing symbolic methods.
The paper tackled the problem of using LLMs for human-centered daily planning, where vague user constraints must be incorporated without strict syntactic requirements. It found that an LLM-based planner (LLMPlan) performed similarly to a symbolic planner in constraint satisfaction (2% difference) but achieved higher user satisfaction (70.5% vs. 40.4%) in evaluations with 40 users.
LLMs have recently made impressive inroads on tasks whose output is structured, such as coding, robotic planning and querying databases. The vision of creating AI-powered personal assistants also involves creating structured outputs, such as a plan for one's day, or for an overseas trip. Here, since the plan is executed by a human, the output doesn't have to satisfy strict syntactic constraints. A useful assistant should also be able to incorporate vague constraints specified by the user in natural language. This makes LLMs an attractive option for planning. We consider the problem of planning one's day. We develop an LLM-based planner (LLMPlan) extended with the ability to self-reflect on its output and a symbolic planner (SymPlan) with the ability to translate text constraints into a symbolic representation. Despite no formal specification of constraints, we find that LLMPlan performs explicit constraint satisfaction akin to the traditional symbolic planners on average (2% performance difference), while retaining the reasoning of implicit requirements. Consequently, LLM-based planners outperform their symbolic counterparts in user satisfaction (70.5% vs. 40.4%) during interactive evaluation with 40 users.