Journal of Political Institutions and Political Economy > Vol 6 > Issue 3–4

Using Generative AI to Calculate Party Positions: A Comparison of Human Experts and Large Language Models

Cantay Caliskan, University of Rochester, USA, cantay.caliskan@rochester.edu , Junhua Huang, University of Rochester, USA, jhuang773@u.rochester.edu , Yiyang Huang, University of Rochester, USA, yhu119@u.rochester.edu , Ruoxuan Lin, University of Rochester, USA, rlin21@u.rochester.edu , Wanting Shan, University of Rochester, USA, wshan2@u.rochester.edu
 
Suggested Citation
Cantay Caliskan, Junhua Huang, Yiyang Huang, Ruoxuan Lin and Wanting Shan (2025), "Using Generative AI to Calculate Party Positions: A Comparison of Human Experts and Large Language Models", Journal of Political Institutions and Political Economy: Vol. 6: No. 3–4, pp 301-327. http://dx.doi.org/10.1561/113.00000126

Publication Date: 01 Oct 2025
© 2025 C. Caliskan, J. Huang, Y. Huang, R. Lin, and W. Shan
 
Subjects
Political economy,  Political parties,  Political psychology
 
Keywords
Generative AIideological biasparty politicsparty systemsLLMs in politicspolitical methodology
 

Share

Download article
In this article:
Introduction 
Data and Methodology 
Empirical Strategy 
Results 
Discussion 
References 

Abstract

The spatial theory of voting, first introduced by Downs (1957), posits that voters choose political parties based on their positions on the political spectrum, from left to right. Traditionally, left-leaning voters favor progressive policies and government intervention, while right-leaning voters support conservative values and free markets. Centrists seek a balance. This framework helps voters find parties that align with their beliefs and preferences. One important tool that voters may use to decide spatially is the Manifesto Project, which analyzes political parties’ election manifestos to study their policy preferences and estimate their positions. This paper investigates the feasibility of replacing human intervention in estimating party positions with intelligent tools, such as LLMs. By examining the party positions estimated by five different LLMs (ChatGPT 3.5 Turbo, Cohere Command, Gemini 1, Llama 2, and Llama 3), our work provides a renewed look at party positions in three major democracies with two-party and multi-party systems (Germany, the United Kingdom, and the United States) and discusses the feasibility of employing AI tools to assist human experts. Results show that, on average, LLMs perceive leftist and rightist manifestos as 73.64% less extreme than they truly are. Given that CMP-trained human experts are likely performing their tasks accurately, this significant bias implies that replacing human experts with LLMs for the calculation of party positions may not be feasible in the near future.

DOI:10.1561/113.00000126

Online Appendix | 113.00000126_app.pdf

This is the article's accompanying appendix.

DOI: 10.1561/113.00000126_app

Companion

Journal of Political Institutions and Political Economy, Volume 6, Issue 3-4 Special Issue: Artificial Intelligence and the Study of Political Institutions
See the other articles that are part of this special issue.