LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selection
This addresses the bottleneck of formalizing complex preferences for human experts in multi-objective decision-making, though it appears incremental as an application of LLMs to an existing problem.
The paper tackles the problem of selecting optimal options from large sets with multiple competing objectives by introducing LISTEN, a framework that uses LLMs as zero-shot preference oracles guided by natural language priorities. Results show LISTEN-U performs well when preferences align parametrically, while LISTEN-T offers robust performance across diverse tasks like flight booking and shopping.
Human experts often struggle to select the best option from a large set of items with multiple competing objectives, a process bottlenecked by the difficulty of formalizing complex, implicit preferences. To address this, we introduce LISTEN, a framework that leverages a Large Language Model (LLM) as a zero-shot preference oracle, guided only by an expert's high-level priorities in natural language. To operate within LLM constraints like context windows and inference costs, we propose two iterative algorithms: LISTEN-U, which uses the LLM to refine a parametric utility function, and LISTEN-T, a non-parametric method that performs tournament-style selections over small batches of solutions. Evaluated on diverse tasks including flight booking, shopping, and exam scheduling, our results show LISTEN-U excels when preferences are parametrically aligned (a property we measure with a novel concordance metric), while LISTEN-T offers more robust performance. This work explores a promising direction for steering complex multi-objective decisions directly with natural language, reducing the cognitive burden of traditional preference elicitation.