GNNs as Predictors of Agentic Workflow Performances
This work addresses the inefficiency in optimizing agentic workflows for users of LLMs, proposing a novel direction for automation, though it is incremental as it applies existing GNN methods to a new domain.
The paper tackles the problem of costly and inefficient optimization of agentic workflows invoked by Large Language Models (LLMs) by proposing Graph Neural Networks (GNNs) as efficient predictors of workflow performances, avoiding repeated LLM invocations, and constructs FLORA-Bench as a benchmarking platform to empirically show that GNNs are simple yet effective predictors.
Agentic workflows invoked by Large Language Models (LLMs) have achieved remarkable success in handling complex tasks. However, optimizing such workflows is costly and inefficient in real-world applications due to extensive invocations of LLMs. To fill this gap, this position paper formulates agentic workflows as computational graphs and advocates Graph Neural Networks (GNNs) as efficient predictors of agentic workflow performances, avoiding repeated LLM invocations for evaluation. To empirically ground this position, we construct FLORA-Bench, a unified platform for benchmarking GNNs for predicting agentic workflow performances. With extensive experiments, we arrive at the following conclusion: GNNs are simple yet effective predictors. This conclusion supports new applications of GNNs and a novel direction towards automating agentic workflow optimization. All codes, models, and data are available at https://github.com/youngsoul0731/Flora-Bench.