Journal of Political Institutions and Political Economy > Vol 6 > Issue 3–4

Divergence or Dysfunction? Using LLMs to Measure Disagreement on the U.S. Supreme Court

Douglas Rice, University of Massachusetts Amherst, USA, drrice@legal.umass.edu , Christopher Zorn, Pennsylvania State University, USA, zorn@psu.edu
 
Suggested Citation
Douglas Rice and Christopher Zorn (2025), "Divergence or Dysfunction? Using LLMs to Measure Disagreement on the U.S. Supreme Court", Journal of Political Institutions and Political Economy: Vol. 6: No. 3–4, pp 329-354. http://dx.doi.org/10.1561/113.00000127

Publication Date: 01 Oct 2025
© 2025 D. Rice and C. Zorn
 
Subjects
Law and economics,  American political development,  Courts,  Judiciary,  Law
 
Keywords
U.S. Supreme Courtjudicial decision-makinglarge language modelstext-as-data
 

Share

Download article
In this article:
Introduction 
The Value of Disagreement 
Measurement Pipeline 
Data 
Results and Analysis 
Discussion and Conclusion 
References 

Abstract

For institutional decision-making bodies like the U.S. Supreme Court, interpersonal dynamics are central to the decision-making process, and the ability of even minimally empowered leaders is crucial in managing those dynamics. Prior research has extensively explored these questions, but measurement difficulties have limited scholarly consensus. Here, we propose a novel methodological approach to these questions, leveraging large language models (LLMs) to score the sentiment of judicial opinions, with a specific focus on differentiating normal jurisprudential disagreements from more personal/targeted criticisms within the Court’s opinions. Employing LLMs, we are able to differentiate between general and targeted criticism, then employ targeted criticism as a variant of aspect-level sentiment analysis over all majority, concurring, and dissenting opinions issued by the Court between 1954 and 2010. In doing so, we develop a measure of the degree of consensual and discordant content for each opinion, justice, and term of the Court. Our approach addresses the limitations of prior methods in distinguishing between what are generally regarded as healthy and unhealthy disagreements, and examining data at the opinion level provides further leverage on uncovering leadership effects. Our results provide new evidence of the chief justice’s role in building and maintaining consensus on the Court and suggest a historically unique modern Court increasingly willing to publicly express disagreement, with implications for understanding institutional debates more generally.

DOI:10.1561/113.00000127

Companion

Journal of Political Institutions and Political Economy, Volume 6, Issue 3-4 Special Issue: Artificial Intelligence and the Study of Political Institutions
See the other articles that are part of this special issue.