Generalized Priority-Aware Shapley Value
For researchers using Shapley value for valuation in machine learning, this work extends priority-aware valuation to real-world cyclic and weighted priority structures, though the impact is incremental as it generalizes existing concepts.
The paper introduces the generalized priority-aware Shapley value (GPASV) for arbitrary directed weighted priority graphs, addressing the limitation of existing methods that require binary acyclic priorities. Applied to LLM ensemble valuation on the cyclic Chatbot Arena preference graph, GPASV reveals that different balances of graph priority versus individual soft priority yield substantively different valuations.
Shapley value and its priority-aware extensions are widely used for valuation in machine learning, but existing methods require pairwise priority to be binary and acyclic, a restriction spectacularly violated in real-data examples such as aggregated human preferences and multi-criterion comparisons. We introduce the generalized priority-aware Shapley value (GPASV), a random order value defined on arbitrary directed weighted priority graphs, in which pairwise edges penalize rather than forbid order violations. GPASV covers a range of classical models as boundary cases. We establish GPASV through an axiomatic characterization, develop the associated computational methods, and introduce a priority sweeping diagnostic extending PASV's. We apply GPASV to LLM ensemble valuation on the cyclic Chatbot Arena preference graph, illustrating that priority-aware valuation is not a one-button operation: different balances of pairwise graph priority versus individual soft priority produce substantively different valuations of the same data.