Human-Centric Autonomous Systems With LLMs for User Command Reasoning
This work addresses the need for human-centric autonomous systems to accurately understand user intent, particularly in complex scenarios, though it appears incremental in applying existing LLM methods to a new domain.
The paper tackles the problem of interpreting user commands in autonomous driving by using Large Language Models (LLMs) to infer system requirements from natural language, achieving results through experiments on few-shot multivariate binary classification accuracy.
The evolution of autonomous driving has made remarkable advancements in recent years, evolving into a tangible reality. However, a human-centric large-scale adoption hinges on meeting a variety of multifaceted requirements. To ensure that the autonomous system meets the user's intent, it is essential to accurately discern and interpret user commands, especially in complex or emergency situations. To this end, we propose to leverage the reasoning capabilities of Large Language Models (LLMs) to infer system requirements from in-cabin users' commands. Through a series of experiments that include different LLM models and prompt designs, we explore the few-shot multivariate binary classification accuracy of system requirements from natural language textual commands. We confirm the general ability of LLMs to understand and reason about prompts but underline that their effectiveness is conditioned on the quality of both the LLM model and the design of appropriate sequential prompts. Code and models are public with the link \url{https://github.com/KTH-RPL/DriveCmd_LLM}.