Journal of Political Institutions and Political Economy > Vol 6 > Issue 2

Measuring Executive Agency Ideology Using Large Language Models

Nicholas G. Napolio, Assistant Professor, University of California, Riverside, USA, nnapolio@ucr.edu
 
Suggested Citation
Nicholas G. Napolio (2025), "Measuring Executive Agency Ideology Using Large Language Models", Journal of Political Institutions and Political Economy: Vol. 6: No. 2, pp 161-180. http://dx.doi.org/10.1561/113.00000121

Publication Date: 07 Jul 2025
© 2025 N. G. Napolio
 
Subjects
Latent variable models,  Bureaucracy,  Public administration
 
Keywords
American politicsbureaucratic politicsideologymeasurementlarge language models
 

Share

Download article
In this article:
Measuring Executive Ideology Using LLMs 
Results: Chat Scores are Valid Measures 
Conclusion and Limitations 
Appendix 
References 

Abstract

I scale executive agencies along a liberal–conservative dimension using ChatGPT, a large language model, to produce both static measures of executive agency ideology and dynamic measures spanning 1949–2020, almost five decades more coverage than existing dynamic measures. My measure correlates strongly with existing measures although does not perfectly replicate them, indicating the measure is not simply regurgitating existing ones. In addition, the new measure predicts congressional delegation to agencies from 1949 to 2018 consistent with the “ally principle” that political actors delegate to agents closest to them in ideological space. The approach adopted here has an advantage over existing ones because my approach produces measures that are cheaper (~$50) and faster (~4 hours of computing time) to produce, that go back four decades earlier than existing measures, and that can easily be scaled up to include more agencies or time periods without much added cost.

DOI:10.1561/113.00000121