Blind Judgement: Agent-Based Supreme Court Modelling With GPT
This work addresses the problem of modeling politically-charged discourse for researchers using language models, though it appears incremental as it applies existing methods to a new domain.
The paper tackled simulating US Supreme Court rulings from 2010-2016 using a Transformer-based multi-agent system trained on justices' opinions, achieving better-than-random accuracy on 96 real cases and finding a correlation between model accuracy and justices' political alignment.
We present a novel Transformer-based multi-agent system for simulating the judicial rulings of the 2010-2016 Supreme Court of the United States. We train nine separate models with the respective authored opinions of each supreme justice active ca. 2015 and test the resulting system on 96 real-world cases. We find our system predicts the decisions of the real-world Supreme Court with better-than-random accuracy. We further find a correlation between model accuracy with respect to individual justices and their alignment between legal conservatism & liberalism. Our methods and results hold significance for researchers interested in using language models to simulate politically-charged discourse between multiple agents.