An approach for systematic decomposition of complex llm tasks
This work addresses the problem of unreliable LLM performance on complex tasks for AI researchers and practitioners, offering a novel approach that is more systematic than existing heuristic methods.
The paper tackles the reliability issues of Large Language Models on complex tasks by introducing ACONIC, a systematic decomposition framework that models tasks as constraint problems and uses formal complexity measures to guide decomposition, resulting in performance improvements of 10-40 percentage points on combinatorial and database querying tasks.
Large Language Models (LLMs) suffer from reliability issues on complex tasks, as existing decomposition methods are heuristic and rely on agent or manual decomposition. This work introduces a novel, systematic decomposition framework that we call Analysis of CONstraint-Induced Complexity (ACONIC), which models the task as a constraint problem and leveraging formal complexity measures to guide decomposition. On combinatorial (SATBench) and LLM database querying tasks (Spider), we find that by decomposing the tasks following the measure of complexity, agent can perform considerably better (10-40 percentage point).