The political consequences of economic globalization has lately been fiercely debated across Europe and the United States, including the role of labor immigration. In this paper we study the party choices of voters facing labor market competition from immigration. To identify the effect of labor market competition we introduce the national skill cell approach, which is designed to isolate a direct partial effect of immigrant competition. By access to detailed, population-wide, administrative data, we get precise measures of Norwegian voters' exposure to competition, and we relate this measure to voting behavior in five national elections. We find a polarizing effect of immigration among voters experiencing negative wage effects of immigration. The polarization points to the existence of a protectionist and a compensatory response, and we propose that predetermined ideological convictions determine the response.