Implementing Rational Choice Functions with LLMs and Measuring their Alignment with User Preferences
This work addresses alignment issues for users of LLM-based decision-making agents, though it is incremental as it builds on existing ranking methods.
The paper tackles the problem of measuring alignment of large language models (LLMs) with user preferences in decision-making contexts, by generalizing existing ranking methods to include strict preferences and indifference, and demonstrates its approach in an automotive intelligent user interface application.
As large language models (LLMs) become integral to intelligent user interfaces (IUIs), their role as decision-making agents raises critical concerns about alignment. Although extensive research has addressed issues such as factuality, bias, and toxicity, comparatively little attention has been paid to measuring alignment to preferences, i.e., the relative desirability of different alternatives, a concept used in decision making, economics, and social choice theory. However, a reliable decision-making agent makes choices that align well with user preferences. In this paper, we generalize existing methods that exploit LLMs for ranking alternative outcomes by addressing alignment with the broader and more flexible concept of user preferences, which includes both strict preferences and indifference among alternatives. To this end, we put forward design principles for using LLMs to implement rational choice functions, and provide the necessary tools to measure preference satisfaction. We demonstrate the applicability of our approach through an empirical study in a practical application of an IUI in the automotive domain.