RWP20 002 Norris
RWP20 002 Norris
RWP20 002 Norris
Pippa Norris
Harvard Kennedy School
February 2020
RWP20-002
www.hks.harvard.edu
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Pippa Norris
McGuire Lecturer in Comparative Politics
John F. Kennedy School of Government
Harvard University
Cambridge, MA 02138
Pippa_Norris@Harvard.edu
www.pippanorris.com
@PippaN15
www.GlobalPartySurvey.org
Data: https://dataverse.harvard.edu/dataverse/GlobalPartySurvey.
1
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
2
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
namely that: (i) the only legitimate authority flows directly from the ‘people’,
and by contrast (ii) the enemy of the people are the ‘establishment’ who are
corrupt, out of touch, and self-serving, false, betraying the public trust, and
thwarting the popular will. The notions of ‘people’ and ‘establishment’ are
fuzzy in this language rather than precise, and malleable to each context and
audience rather than well-defined.
In the political sphere, the rhetoric claims that legitimacy lies with the
‘people’, a powerful claim echoing core democratic values. In practice,
however, the language may often be weaponized as a façade by strongman
leaders with authoritarian values who claim a direct mandate to act on behalf
of ‘the people’ while they actively seek to dismantle checks and balances on
executive power. This includes attacking the legitimacy of elected
representatives and political opponents, as well as the courts, judges, and rule
of law, state officials and mainstream media, along with the broader range of
policy technocrats, professional think-tanks, academic opinion-formers, and
scientific consultants.
As argued in our previous work (Norris and Inglehart 2019):
“In this regard, populism is treated not a distinct type of leadership, or
even a family of political parties, as is often assumed, but rather as a
discourse about governance that can be adopted by actors across the
entire ideological spectrum. We reject the notion that populism, in
itself, makes other ideological claims about substantive or
programmatic claims about what should be done; instead it is a rhetoric
about the rightful location of governance authority in any society.
In Western democracies, the most common antithesis of populism is
‘Pluralism’, emphasizing the importance of tolerating multiculturalism
and social diversity in society, governance through liberal democratic
institutions, the role of checks and balances on executive powers, and
respect for minority rights to counterbalance the majoritarian voice of
the people. In non-democratic countries, however, populism may also
be contested by those advocating ‘elitism’ – claiming that power should
rest in the hands of a single leader, a leadership elite, or a predominant
party.” (Norris and Inglehart 2019).
Ideologies such as socialism, communism or liberalism present a set of
coherent values and principles, suggesting a plan of action to achieve these
goals, with policies on a wide range of issues like the economy, immigration,
the environment, and international affairs. By contrast, populism does not
provide a vision about the good society or present a coherent set of ideas.
The chameleonic rhetoric of populism is adaptable to parties and leaders of
many political persuasions. It is colored by ‘second order’ principles,
including varied values and issue positions.
The European literature has conventionally categorized populists as part of
the so-called ‘radical right’ or ‘extreme right’, implying unidimensional
patterns of party competition (Ignazi 2003; Betz 1994; Betz and Immerfall
1998; Hainsworth 1992, 2000; Mudde 2007, 2016; Wodak et al. 2013). But,
in fact, a broader perspective suggests that these labels, and indeed even the
conventional Left-Right framework, are often inadequate to capture the
varied types of populists found around the world. Many populist leaders have
traditionally been regarded as economically leftwing in Central and Eastern
3
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Europe (Pirro 2015) as well as in Latin America (Torre and Arnson 2013).
In the United States, as well, President Trump advocates a mélange of policies
including America-First nationalism, anti-immigrant nativism, and social
conservatism on cultural issues like gay rights, climate change, gun control,
and reproductive rights, mixed with protectionist economic policies like
tariffs and farm subsidies which fly in the face of traditional GOP orthodoxy.
The populist tropes can also be heard in the language of social democrats like
Senator Bernie Sanders and Elizabeth Warren, railing against the billionaire
class and policies benefitting the ‘few not the many’ while advocating
progressive taxation and healthcare for all.
One reason for the difficulties of understanding this phenomenon is that, like
an architectural façade for a movie studio lot, the rhetorical style (and thus
the ideas being expressed in the speech) may be genuine or faux. The
language is chameleonic and adaptable. Slogans are intended to signal shared
meanings recognized by its audience as largely symbolic rather than literal
(‘Build the Wall’, ‘Lock her Up’, ‘Drain the Swamp’, ‘Get Brexit Done’).
Critics regard such slogans as transparent lies. By expressing shared feelings
of anger, however, supporters may believe that the words reflect a deeper
truth (Wodak 2015). The exaggerative hyperbole anticipates the hopes and
fears of the audience, building repetitive call-and-response choruses
interacting with the speaker, as predictable as familiar church rituals. Indeed,
the rhetoric may say more about the audience than the speaker. President
Trump, for example, uses two distinct styles of oratory, one flat and
humdrum, read from the teleprompter for official occasions, the other
impromptu, rambling, repetitive, and deeply emotive, tapping into the
impotent rage, victimhood resentment, and frustration of his base (Kelly
2019; Rowland 2019; Lamont, Park and Ayala-Hurtado 2017). What version
is real? Whether language conveys the genuine intentions of any speaker
cannot be determined by analysis of the meaning of the words alone, but only
by whether their actions are either consistent with the language – or
contradictory. In the words of President Obama: “Now suddenly President
Trump acts like he’s a populist helping working people? Come on, man. You wanna know
what somebody’s gonna do? Look at what they have been doing their whole lives.”
[Figure 1 about here]
This understanding leads to the framework illustrated in Figure 1. The use of
populist rhetoric is conceptualized as a ‘first order’ set of normative values
about the rightful distribution of legitimate authority and power in decision-
making processes. In liberal democracies, the most common contrast to
populism is the rhetoric of ‘liberal pluralism’, emphasizing the values of
multiculturalism and tolerance of social diversity, the importance of liberal
democratic institutions in providing checks and balances limiting the abuse
of executive powers, and respect for minority rights to counterbalance the
majoritarian voice of ‘the people’. The substantive meaning of the rhetoric
about governance becomes colored by the second order principles, including
social and economic values and programmatic policy positions, reflecting
core socioeconomic and cultural cleavages in the mass electorate.
II: Measuring party values and rhetoric
Until relatively recently, an extensive literature on populism in Latin America
has most commonly drawn upon intensive case-studies of particular leaders
4
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
What are the pros and cons of the main approaches for identifying patterns
of party competition -- and which are potentially most useful for populism
studies?
5
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
and ‘Greens’ (Ware 1996; Mair and Mudde 1998). The titles that parties adopt
goes to the heart of their brand name in the political marketplace and their
historical origins. It seems a straightforward approach to classify party
families by their self-adopted names.
In practice, however, party labels can disguise deep ideological divisions, such
as those between neo-classical laissez-faire liberals, favoring free markets and
a minimal role for governments, versus social liberals, advocating cradle-to-
grave welfare states, progressive taxation, and generous public services.
Moreover, identical party labels have been adopted by parties with
contrasting platforms and ideologies. The same party name can also mask
major ideological shifts over time, exemplified by the contrasts between the
Republican party under George H. Bush and Donald Trump, or the British
Labour party under the leadership of Tony Blair and Jeremy Corbyn. Parties
and leaders widely regarded as populists do not adopt the populist label and
many, wanting to burnish their outsider credentials, also adopt sui generis
names to avoid being associated with established parties, exemplified by the
Italian Five Star Movement, Poland’s Law and Justice Party, the Greek
Golden Dawn, and the Spanish Podemos.
(ii) Transnational party affiliations
Scholars of party politics often seek to classify distinct party families through
common transnational organizational networks, such as membership in the
European People’s Party, or institutional affiliations with transnational
federations, exemplified by the Liberal International and the Global Greens.
This approach is most useful for classifying older party families, such as
members affiliated with the Socialist International. This method is relatively
straightforward to apply using publicly accessible information. It also reflects
the way that political parties seek to build international alliances, cooperate
across borders, and share resources with sister organizations (Gomez-Reino
2017).
But these networks have proved far from stable, with populist parties shifting
alliances; for example, the Italian Northern League belonged to the Rainbow
Coalition in the European Parliament before moving in 1994 to join the
Euro-liberals. Political parties can also be members simultaneously of
different associations at transnational, national, and regional levels. For
example, several parties in the European Parliament which are commonly
regarded as populist are currently associated with the Europe of Nations and
Freedom (ENF) group. This includes the Austrian Freedom Party (FPO), the
Flemish Vlaams Belang (FB), the French National Rally (FN), the Alternative
for Germany (AfD), the Italian Lega Nord, and the Dutch Party for Freedom
(PVV). But the Europe of Freedom and Direct Democracy (EFD) provides
another rival group in the European Parliament, linking UKIP (prior to
Brexit), the Alternative for Germany, the Five Star Movement, the Lithuanian
Order and Justice party, and the Sweden Democrats. The EFD is
Eurosceptic, using populist rhetoric claiming to reflect ‘the people’s voice’ by
fighting ‘big government, big banks, and big business’ which are ‘strangling
national identities.’
Not all European parties commonly seen as populist belong to these two
groupings, however, for example the Danish People’s Party and the Finns
Party are members of the center-right European Conservatives and
6
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
7
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Most attempts at party classification based on this data have used the familiar
‘Left-Right’ dimension, including redistributive economic issues reflecting
the class cleavage, such as party positions for or against taxation, welfare
spending, privatization or nationalization, and so on. These were key to
patterns of party competition in many European countries during the post-
war era with Keynesian economic policies, pro-welfare state, and public
ownership, on the center-left, and free-market policies favoring a smaller role
for the state, deregulation, and low taxes on the center-right. The CMP data
demonstrates that the Left-Right issue cleavage has gradually faded in
importance, however, as party programs during recent decades have often
given greater prominence to Liberal and Conservative social issues, like
LGBTQ rights, immigration, reproductive rights, and environmental
protection, reflecting contemporary culture wars (Norris and Inglehart 2019).
The CPM project has become the standard reference source for analyzing
trends in the issue agenda and party competition in European democracies,
where parliamentary parties publish official manifesto documents which
guide their policy programs if they are elected to power. Outside of this
context, however, platforms may not function as important guides to party
policy, for example, in presidential contests with personalistic competition,
where parties compete with clientelist appeals, or where parties and party
systems are unstable and poorly institutionalized. Even more importantly,
the CMP coding system was not designed to capture the core components of
populism as a style of discourse -- and it is difficult to see how the data could
be adapted retrospectively for this purpose
(v) Expert surveys
Expert surveys have been increasingly adopted within the global scientific
and policy communities to construct multiple international and domestic
indicators of complex phenomena, exemplified by the World Bank Institute
Good Governance indices, the Varieties of Democracy project,
Transparency International’s Corruption Perception Index, and the Electoral
Integrity Project (Cooley and Snyder 2015).
The technique has been used for a long series of comparative studies seeking
to classify the ideological values and issue positions of political parties based
on expert estimates, including datasets produced by Castles and Mair (1984),
Laver and Hunt (1992), Huber and Inglehart (1995), Ray (1999),
Wiesehomier (2019), and the series of Chapel Hill Expert Survey (CHES)
conducted every four years since 1999 (Hooghe et al 2010; Bakker et al. 2012,
2015).
The series of Chapel Hill Expert Survey (CHES) has traditionally focused on
measuring party competition in Europe towards the classic Left-Right
ideological cleavage in post-war politics, monitored by party positions on
policy issues such as management of the economy, social welfare, and
European integration, as well as the Liberal-Conservative cleavage revolving
around cultural and social values, such as positions towards issues of
immigration, environmental protection, reproductive rights, and
homosexuality. CHES was not designed to measure populism, however,
although selected anti-elite and anti-corruption variables from the 2014
questionnaire have been adapted for this purpose (Inglehart and Norris 2016;
Polk 2017; Simmons et al 2018; Norris and Inglehart 2019).
8
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Two other recent attempts have sought to fill this lacuna by identifying cross-
national lists of populist parties. Team Populism worked with journalists at
The Guardian, consulting an informal network of 30 scholars to generate a list
of 127 political parties classified as populist/ non-populist in thirty Europe
states (Rooduijn et al 2019). In another study, for the Tony Blair Institute,
Kyle and Gultchin (2018) developed a database of 53 populist leaders in 33
countries by identifying the use of the term through searches in the academic
journal literature. These classifications have not been subject to robustness
tests, however, and the use of simple binary categories (populist/non-
populist) makes them vulnerable to major classification errors.
III: THE GPS RESEARCH DESIGN
To address these issues, the Global Party Survey, 2019 (GPS) has been
designed as an international expert survey to compare key ideological values,
issue positions, and populist rhetoric used by political parties around the
world. The questionnaire includes 21 core survey items designed to estimate
key ideological values, issue positions, and populist rhetoric. The dataset
provides estimates for 1,052 parties in 163 countries, drawing on responses
gathered from 1,861 party and election experts. There are several potential
advances from the new evidence, namely the Global Party Survey:
• Expands comparisons of political parties well beyond Western
democracies;
• Uses fine-grained scaled measures which can be disaggregated into
categories;
• Incorporates standardized party codes for merging with related
cross-national macro, mezzo and micro-level datasets; and,
• The methods maintain continuity with previous studies, facilitating
robustness tests.
Questionnaire design
The questionnaire for the Global Party Survey was administered through the
Qualtrics platform. It includes 21 core items measured using ten-point
continuous scales (illustrated in Figure 1). These were designed to identify
each party’s current position and salience ideological values on Left-Right
economic values and Liberal-Conservative Social Values; their position on
several policy issues such as spending and taxation, immigration, nationalism,
women’s rights, ethnic minority rights, liberal democracy, and environmental
protection; as well as several items monitoring their position and saliency
concerning the use of populist rhetoric. The full GPS Questionnaire and
Codebook can be downloaded from the project website, along with some
data visualizations illustrating the results.
[Figure 2 about here]
As well as the core items, the GPS survey also asked questions about the
expert’s nationality and citizenship, gender, age, party preferences, and their
self-reported L-R ideology, as well as their familiarity with each of the parties,
and the degree of difficulty they experienced in completing the survey. Many
more items were considered but eventually dropped as the final design sought
to establish a judicious trade-off which balanced the length of the
questionnaire with the likely fall in the response rate from using a longer
9
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
10
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
11
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
12
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Experts
Participation in the survey was by personal invitation only. Experts were
defined as scholars of parties and elections selected for each country drawing
upon the global database established since 2012 by the Electoral Integrity
Project, checked and verified according to several criteria. Experts were
defined as political scientists (or other social scientist in a related discipline)
who had demonstrated knowledge of the electoral process and parties in a
particular country, such as through publications, membership of a relevant
research group, or university employment. This pool was supplemented in a
few smaller countries, like island states in the Caribbean, by several additional
scholarly party experts suggested by respondents using the ‘snowball’
technique.
Expert surveys have greatly expanded in use during the last decade (Cooley
and Snyder 2015). Like other approaches, however, there are many sources
of potential bias in the estimates they produce (Budge 2000; Mair 2001;
Martinez i Coma and Van Ham 2015). This includes potential errors of
judgments arising from assessments of complex multidimensional
phenomenon, varied contexts for party competition under different regimes,
and the depth of scholars’ expertise on the topic.
Ever since Almond and Verba’s Civic Culture (1963), one classic challenge
facing cross-national surveys arises from the appropriate benchmarks which
people may employ in making their assessments, for example, whether
current a party position is judged relative to their past location, or compared
with rival parties within a country, or else compared with parties in other
societies. As discussed later, the external validity of the data can be examined
most effectively by comparing the GPS estimates with similar independent
studies of the same parties.
To test for internal validity, the GPS expert-level dataset allows users to
analyze whether estimates of party positions were systematically influenced
by the background and personal characteristics of participants, such as their
nationality, Left-Right ideological leanings, gender, or age (Curini 2009).
Regression analysis models using the individual-level expert dataset suggest
that none of these factors were significant predictors of estimates for the
position of parties on the Pluralism-Populist scale.
Two-thirds of respondents were born in their country of expertise, while
three quarters are currently a citizen of that country. One quarter of the
experts in the survey were female, reflecting gender disparities in the
discipline. Respondents were asked to identify party positions in one country
reflecting their primary area of published expertise, irrespective of their
nationality or institutional location. The survey included both resident
(domestic) and international experts (e.g. a scholar teaching at an American
university who specializes in Egyptian or Liberian politics).
There are questions about the reliability of academic experts, in particular
whether their estimates may be skewed by more liberal personal values. To
test this, the position of experts on the self-reported 10-point Left-Right
ideological scale can be compared. The mean was 4.75, just below the mid-
point illustrated in Figure 3, suggesting a fairly balanced distribution.
[Figures 3 and 4 about here]
13
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Overall, when asked about the difficulty or ease of completing the survey on
a ten-point scale, as illustrated in Figure 4, most reported positively that it
was fairly easy (Mean 7.82). There was usually greater familiarity with larger
parties, however, suggesting caution is advisable when analyzing estimates for
the smaller parties.
Response Rate
Personalized survey invitations were distributed by email to experts, and
responses collected online, through the Qualtrics platform. Fieldwork was
conducted for one month, with an initial personal invitation to individual
experts followed by two reminders, from 19 November to 20 December
2019.
Responses were received from 1,891 experts in total, representing an overall
response rate of 23%. On average, each country included replies from around
a dozen experts, but the numbers varied a great deal.
Like V-Dem, the GPS study aimed to include replies from a minimum of five
experts per Country-Party. Sometimes we received fewer replies, however,
especially in smaller developing societies in Africa and the Middle East; in
states governed by autocratic regimes restricting freedom of expression; and
in countries where fewer political scientists specialize in the study of parties
and elections, and in ‘all of the above’. For example, on average around 19
experts responded to assess parties in liberal democracies. By contrast,
around five expert estimates were gathered in many closed autocracies.
There are obviously tensions between the desire for the broadest global
coverage and the need for considerable caution about the reliability of the
estimates for societies with few responses, generating large confidence
intervals. Users can take several steps with these particular cases.
Firstly, the number of experts per country is included as a variable in the
dataset (Experts). Users may choose to adopt any minimum threshold. For
example, the V-Dem project advises users to drop country cases in their
dataset with three or fewer expert estimates. The variable (Min_experts) can
be selected in GPS to filter out cases below 4.9 experts, removing 193 party
estimates. Users may also choose to filter or weight the estimates based on
several indicators. This includes a ten-point scaled measure of expert
familiarity with each political party in each country, which was included as the
first item in the questionnaire, along with a 0-10 point scaled measure of the
difficulty which experts reported at the end of the questionnaire after
completing the survey. Finally, analysts may choose to aggregate across
country cases, for example to compare world regions or party families.
IV: KEY RESULTS AND ROBUSTNESS TESTS
The dataset contains many variables which can be explored in detail for
macro-level cross-national comparisons of how multidimensional patterns of
party competition vary across different world regions, types of regime,
cultural areas, or levels of development. Figures 5 and 6 provide a brief
illustration of some of the results comparing the key ideological scales
measuring Left-Right economic values and Liberal-Conservative social
values. The size of each party is show by the size of the symbol, and only
parties with more than 10% of the seats in the national parliament are shown
in the graphs, excluding smaller parties. The degree to which political parties
14
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
15
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
16
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
17
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
18
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
their own further comparisons, for example by global region or party type.
At the same time, however, as discussed earlier, there remains a health
warning about the size of the margin of error of the GPS estimates in parties
and countries with few respondents. Replication of the survey in future years
would help to provide further cross-checks on the robustness of the
estimates.
V: CONCLUSIONS AND DISCUSSION
A wealth of popular commentary has sought to understand the rise of
populist parties and the consequences of this development. Unfortunately to
date cross-national measures of populist parties capable of throwing light on
these issues has been limited in a number of ways.
Firstly, comparative studies have devoted most attention to examining parties
and leaders in established democracies, especially in Western Europe, where
populist actors have conventionally been categorized as part of the ‘radical
right’, or the ‘extreme right’, implying a unidimensional form of party
competition. The use of the terms ‘left’ and ‘right’ to categorize parties is
deeply ingrained but it may serve to disguise important contrasts. As
demonstrated in this study, deeply conservative cultural values are indeed
expressed on issues like immigration, gay rights, and gender equality by many
populist parties, like the Polish Justice and Development party, Spain’s Vox,
and the Swiss People’s Party. Yet a global perspective suggests that populist
leaders like Chavez and Sanders, and parties like Fidesz, the Slovak National
Party, and Jobbik, endorse leftwing economic policies. And other populist
parties, like Syriza and Unidos Podemos, endorse progressive or liberal social
values (Duina and Carson 2019). In this regard, scholars need to identify
varieties of populists distributed across a multidimensional issue space, such
as distinguishing Progressives, the Nativist Left, Laissez faire Libertarians, as
well as the Authoritarian Right, rather than assuming that all can be neatly
categized by their ‘extreme’ rightwing position along a single Left-Right
dimension (Norris 2019).
The new dataset still needs much further refinement in future iterations,
particularly by improving the accuracy of the list of political parties in several
countries, especially where there is rapid change in unstable party systems. It
is also important to gather more expert assessments in the most challenging
cases where few responses were received, including in smaller developing
societies and in countries with autocratic regimes. The basic approach could
also be replicated to gather new data to compare political leaders, as well as
political parties. Subsequent surveys can be used as further checks on the
reliability of the estimates.
Nevertheless, despite these qualifications, the new dataset has many potential
advantages for users. In particular:
• To improve measurement, the GPS questionnaire incorporates six
alternative indicators of populist rhetoric, measured by standardized
0-10 point continuous scales. The more granular measure provides
greater precision that simple binary classifications.
• By gauging the use of populist rhetoric by parties, as well as their
adherence to key ideological values, and a range of issue positions,
analysts can develop systematic typologies identifying varieties of
populism.
19
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
20
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Source: www.GlobalPartySurvey.org
21
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
22
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
23
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
24
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Source: www.GlobalPartySurvey.org
25
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Source: www.GlobalPartySurvey.org
26
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Source: www.GlobalPartySurvey.org
27
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
KEY: Left-Right values on the horizontal axis are measured by: “Parties can be classified by their current stance on ECONOMIC ISSUES such as privatization, taxes, regulation, government
spending, and the welfare state. Those on the economic LEFT want government to play an active role in the economy. Those on the economic RIGHT favor a reduced role for government. Where would you place
each party on the following 0-10 scale?” Social values on the vertical axis are measured by: “Parties can also be classified by their current social values. Those with LIBERAL values favor expanded
personal freedoms, for example, on abortion rights, same-sex marriage, and democratic participation. Those with CONSERVATIVE values reject these ideas in favor of order, tradition and stability, believing
that government should be a firm moral authority on social and cultural issues. Where would you place each party on the following 0-10 scale?” Populism, coloring the categorized bubbles, is measured
by: “Parties can also be classified by their current use of POPULIST OR PLURALIST rhetoric. POPULIST language typically challenges the legitimacy of established political institutions and emphasizes that
the will of the people should prevail. By contrast, PLURALIST rhetoric rejects these ideas, believing that elected leaders should govern, constrained by minority rights, bargaining and compromise, as well as checks
and balances on executive power. Where would you place each party on the following 0-10 scale?” Red=high, green=low. The bubble size reflects the % seat for each party in recent elections.
www.GlobalPartySurvey.org
28
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
KEY: Left-Right values on the horizontal axis are measured by: “Parties can be classified by their current stance on ECONOMIC ISSUES such as privatization, taxes, regulation, government spending,
and the welfare state. Those on the economic LEFT want government to play an active role in the economy. Those on the economic RIGHT favor a reduced role for government. Where would you place each party
on the following 0-10 scale?” Social values on the vertical axis are measured by: “Parties can also be classified by their current social values. Those with LIBERAL values favor expanded personal freedoms,
for example, on abortion rights, same-sex marriage, and democratic participation. Those with CONSERVATIVE values reject these ideas in favor of order, tradition and stability, believing that government should
be a firm moral authority on social and cultural issues. Where would you place each party on the following 0-10 scale?” Populism, coloring the categorized bubbles, is measured by: “Parties can also be
classified by their current use of POPULIST OR PLURALIST rhetoric. POPULIST language typically challenges the legitimacy of established political institutions and emphasizes that the will of the people
should prevail. By contrast, PLURALIST rhetoric rejects these ideas, believing that elected leaders should govern, constrained by minority rights, bargaining and compromise, as well as checks and balances on
executive power. Where would you place each party on the following 0-10 scale?” Red=high, green=low. The bubble size reflects the % seat for each party in recent elections.
www.GlobalPartySurvey.org
29
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
KEY: Left-Right values on the horizontal axis are measured by: “Parties can be classified by their current stance on ECONOMIC ISSUES such as privatization, taxes, regulation, government spending,
and the welfare state. Those on the economic LEFT want government to play an active role in the economy. Those on the economic RIGHT favor a reduced role for government. Where would you place each party
on the following 0-10 scale?” Social values on the vertical axis are measured by: “Parties can also be classified by their current social values. Those with LIBERAL values favor expanded personal freedoms,
for example, on abortion rights, same-sex marriage, and democratic participation. Those with CONSERVATIVE values reject these ideas in favor of order, tradition and stability, believing that government should
be a firm moral authority on social and cultural issues. Where would you place each party on the following 0-10 scale?” Populism, coloring the categorized bubbles, is measured by: “Parties can also be
classified by their current use of POPULIST OR PLURALIST rhetoric. POPULIST language typically challenges the legitimacy of established political institutions and emphasizes that the will of the people should
prevail. By contrast, PLURALIST rhetoric rejects these ideas, believing that elected leaders should govern, constrained by minority rights, bargaining and compromise, as well as checks and balances on executive
power. Where would you place each party on the following 0-10 scale?” Only strongly [populist parties (scoring 7.5 or above.) The bubble size reflects the % seat for each party in recent elections.
www.GlobalPartySurvey.org
30
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
9% (20) 5% (9)
31
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Figure 9: Comparing the GPS and CHES expert estimates of Liberal-Conservative social values
Notes: Q: “Parties can also be classified by their current social values. Those with LIBERAL values favor expanded personal freedoms, for example, on abortion rights, same-sex marriage, and
democratic participation. Those with CONSERVATIVE values reject these ideas in favor of order, tradition and stability, believing that government should be a firm moral authority on social and
cultural issues. Where would you place each party on the following scale?”
Sources: www.GlobalPartySurvey.org CHES-2017
32
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Figure 10: Comparing the GPS expert estimates of populism with the Popu-List classification
Notes: “We seek to understand the type of rhetoric commonly used by each party, such as in their leadership speeches, rallies, press releases, party platforms, and campaign
communications. Vertical Axis: Parties can be classified by their current use of POPULIST OR PLURALIST rhetoric. POPULIST language typically challenges the legitimacy of established political
institutions and emphasizes that the will of the people should prevail. By contrast, PLURALIST rhetoric rejects these ideas, believing that elected leaders should govern, constrained by minority
rights, bargaining and compromise, as well as checks and balances on executive power. Where would you place each party on the following 0-10 scales? And how important is populist rhetoric for
each of these parties?” Parties in RED are identified as populist by the Popu-List project.
Source: https://popu-list.org/ www.GlobalPartySurvey.org
33
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Figure 11. Comparing the GPS expert estimates of Left-Right party positions with the mean Left-Right position of each party’s voters in Liberal
Democracies
Note:
KEY:OECD countries
Left-Right valuesonly. Vertical
of parties axis=WVS_LR_PARTYVOTER.
on the horizontal axis are measured in theHorizontal Axis=V4_scale.
GPS by: “Parties can be classified Colors by Type_Values
by their current Crimson ISSUES
stance on ECONOMIC =Left-Liberal/ Umber=Left-
such as privatization, taxes,
Conservative/ Azure=Right-Liberal/ Royal Blue=Right-Conservative.
regulation, government spending, and the welfare state. Those on the economic LEFT want government to play an active role in the economy. Those on the economic RIGHT favor a reduced role for
government.www.GlobalPartySurvey.org
Sources: Where would you place each partyand
on the following 0-10
WVS/EVS scale?” LR position of each party’s voters on the vertical axis are measured in the WVS/EVS 1-7 by:“ “In political
1-7.
matters, people talk of ‘the left’ and ‘the right.’ How would you place your views on this scale, generally speaking.” Coded Left (1) to Right (10). Populism, coloring the categorized
bubbles, is measured by: “Parties can also be classified by their current use of POPULIST OR PLURALIST rhetoric. POPULIST language typically challenges the legitimacy of established political
institutions and emphasizes that the will of the people should prevail. By contrast, PLURALIST rhetoric rejects these ideas, believing that elected leaders should govern, constrained by minority rights, bargaining
and compromise, as well as checks and balances on executive power. Where would you place each party on the following 0-10 scale?” Red=high, green=low. The bubble size reflects the % seat for each
party in recent elections. Liberal democracies only. www.GlobalPartySurvey.org and www.WorldValuesSurvey.org
34
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Acknowledgments
The survey was directed by Pippa Norris at Harvard and administered by Laura Welty
at the University of Sydney. It was funded by the Australian Research Council
Laureate award as part of the Electoral Integrity Project. The research design was
developed in conjunction with colleagues at the Chapel Hill Expert Survey (CHES),
Liesbet Hooghe, Gary Marks and Ryan Bakker, the team conducting a similar survey
in Europe and Latin America. The study also benefited from several other scholars
of parties and elections who were consulted about the early draft survey design and
who generously offered invaluable suggestions, including Mark Franklin, David
Farrell, Ioannis Andreadis, Tim Bale, and Susan Scarrow. Comments and corrections
remain welcome for subsequent releases of the dataset. Finally, the research could
not have been completed without the time and effort which all the experts invested
in completing the survey – which is greatly appreciated.
The Global Party Survey dataset (Version 1, Spring 2020) is available in Excel, SPSS,
Stata and .csv formats at the level of experts and parties, along with the Codebook
and Questionnaire. Details are available from the project website
www.GlobalPartySurvey.org and the data files can be downloaded from:
https://dataverse.harvard.edu/dataverse/GlobalPartySurvey
References
Aalberg, Toril, Frank Esser, Carsten Reinemann, Jesper Stromback, and
Claes H. de Vreese. Eds. (2017). Populist Political Communication in
Europe. London: Routledge.
Akkerman A., Cas Mudde and A. Zaslove. (2014). ‘How populist are the
people? Measuring populist attitudes in voters.’ Comparative Political
Studies, 47(9), 1324–1353
Bailer, S. (2014). ‘Interviews and surveys in legislative research.’ In S. Martin,
T. Saalfeld, and K. Strom. (eds) The Oxford Handbook of Legislative
Studies. Oxford: Oxford University Press.
Bakker, Ryan et al. (2012). ‘Measuring party positions in Europe: The Chapel
Hill expert survey trend file, 1999–2010.’ Party Politics 21(1).
Bakker, Ryan, Ryan Bakker, Catherine de Vries, Erica Edwards, Liesbet
Hooghe, Seth Jolly, Gary Marks, Jonathan Polk, Jan Rovny, Marco
Steenbergen, and Milada Vachudova. (2015). ‘Measuring party
positions in Europe: The Chapel Hill expert survey trend file, 1999-
2010.’ Party Politics 21(1): 143-152.
Benoit, Kenneth and Michael Laver. (2006) Party Policy in Modern
Democracies. London: Routledge.
Betz, Hans-George and Stefan Immerfall. Eds. (1998). The New Politics of the
Right: Neo-Populist Parties and Movements in Established Democracies. New
York: St Martin's Press.
Betz, Hans-George. (1994). Radicalism and Right-wing populism in Western Europe.
New York: St. Martin’s Press.
Bonikowski, Bart and Noam Gidron. (2016). ‘The populist style in American
politics.’ Social Forces 94(4): 1593–1621.
Budge, Ian, Hans-Dieter Klingemann, A. Volkens, Judith Bara, and Eric
Tanenbaum. (2001) Mapping Policy Preferences: Estimates for Parties,
Electors, and Governments 1945–1998. NY: Oxford University Press.
35
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Budge, Ian. (2000) ‘Expert judgments of party policy positions: Uses and
limitations in political research’, European Journal of Political Research
37(1): 103-13.
Budge, Ian and Dennis J. Farlie. (1983). Explaining and predicting elections: issue
effects and party strategies in twenty-three democracies. London: Allen and
Unwin.
Castles, Frank and Peter Mair (1984) ‘Left-Right Political Scales: Some
Expert Judgements’, European Journal of Political Research 12: 73-88.
Curini, Luigi. (2009) ‘Experts’ Political Preferences and Their Impact on
Ideological Bias: An Unfolding Analysis based on a Benoit-Laver
Expert Survey.’ Party Politics 16(3)
https://doi.org/10.1177/1354068809341051
Doring, Herbert and Sven Regel. (2019). ‘Party Facts: A database of political
parties worldwide.’ Party Politics 25(2): 97-109.
Duina, Francesco and Dylan Carson. (2019). ‘Not so right after all? Making
sense of the progressive rhetoric of Europe’s far-right parties.’
International Sociology 35(1): 3-21.
Gerring, John. (2004.) ‘What is a case study and what is it good for?’ American
Political Science Review, 98(2): 341-354.
Hainsworth, Paul. Ed. (1992). The Extreme Right in Europe and the USA. New
York: St Martin's Press.
Hainsworth, Paul. Ed. (2000). The Politics of the Extreme Right: From the Margins
to the Mainstream. London: Pinter.
Hawkins, Kirk A. (2009). ‘Is Chávez populist? Measuring populist discourse
in comparative perspective’ Comparative Political Studies, 42(8): 1040-
1067.
Hawkins, Kirk A., Rosario Aguilar, Bruno Castanho Silva, Erin K. Jenne,
Bojana Kocijan, and Cristóbal Rovira Kaltwasser (2019b), Global
Populism Database, v1 https://doi.org/10.7910/DVN/LFTQEZ,
Harvard Dataverse, V1,
Hawkins, Kirk, Ryan E. Carlin, Levente Littvay and Cristóbal Rovira
Kaltwasser. Eds. (2019). The ideational approach to populism: concept,
theory, and analysis. New York: Routledge.
Hooghe, Liesbet, Gary Marks, Carole J. Wilson. (2002). ‘Does Left/Right
structure party positions on European integration? Comparative
Political Studies 35(8).
Hooghe, Liesbet, Ryan Bakker and A. Brigevich. (2010) ‘Reliability and
validity of measuring party positions: The Chapel Hill Expert
Surveys of 2002 and 2006.’ European Journal of Political Research 49(5):
687–703.
Huber, John and Ronald Inglehart (1995) ‘Expert Interpretations of party
space and party locations in 42 Societies.’ Party Politics 1: 73-111.
Ignazi, Piero. (2003). Extreme right parties in Western Europe. New
York: Oxford University Press.
36
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Inglehart, Ronald and Pippa Norris. (2016). ‘Trump, Brexit, and the Rise of
Populism: Economic Have-Nots and Cultural Backlash.’ HKS Working
Paper RWP16-026.
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2818659
Jagers, J. and Stephan Walgrave. (2007). ‘Populism as political
communication style: An empirical study of political parties’
discourse in Belgium.’ European Journal of Political Research 46(3):
319–345.
Katz, Richard S. and Bernhard Wessels (eds) (1999) The European Parliament,
the National Parliaments, and European Integration. Oxford: Oxford
University Press.
Kelly, Casey Ryan. (2019) ‘Donald J. Trump and the rhetoric of
resentment.’ Quarterly Journal of Speech.
Keman, Hans (2007) ‘Experts and Manifestos: Different Sources - Same
Results for Comparative Research?’, Electoral Studies 26: 76-89.
Klingemann, Hans-Dieter, Andrea Volkens, Judith Bara, Ian Budge, Michael
McDonald. (2006). Mapping Policy Preferences II: Estimates for Parties,
Electors, and Governments in Eastern Europe, European Union and OECD
1990–2003. Oxford University Press, Oxford.
Kreis, Ramona. 2017. ‘The “Tweet Politics” of President Trump.’ Journal of
Language and Politics 16(4): 607-618.
Krouwel, André and Annemarie van Elfrinkhof. (2014). ‘Combining
strengths of methods of party positioning to counter their
weaknesses.’ Quality & Quantity 48: 1455-1472.
Lamont, Michele, Bo Yun Park, and Elena Ayala-Hurtado. (2017). ‘Trump's
electoral speeches and his appeal to the American white working
class.’ British Journal of Sociology 68: 153-180.
Laver, Michael and Ben Hunt. (1992) Policy and Party Competition. London:
Routledge.
Laver, Michael. (2000). ‘Estimating policy positions from political texts.’
American Journal of Political Science 44(2): 619-634.
Laver, Michael. (ed.) (2001). Estimating the Policy Positions of Political Actors.
London: Routledge.
Mair, Peter (2001) ‘Searching for the Positions of Political Actors: A Review
of Approaches and a Critical Evaluation of Expert Surveys.’ In
Michael Laver (ed) Estimating the Policy Position of Political Actors,
pp. 10-30. New York: Routledge.
Mair, Peter and Cas Mudde. (1998). ‘The party family and its study.’ Annual
Review of Political Science,1: 211–29.
Marks, Gary, Hooghe, Liesbet, Steenbergen, Marco R. and Bakker, Ryan.
(2007). ‘Cross-validating data on party positioning on European
integration.’ Electoral Studies 26 (1): 23-38.
Martinez I Coma, Ferran and Carolien Van Ham. (2015). ‘Can experts judge
elections? Testing the validity of expert judgments for measuring
election integrity.’ European Journal of Political Research 54(2): 305-325.
37
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
Mudde, Cas. Ed. (2016). The Populist Radical Right: A Reader. London:
Routledge.
Mudde, Cas. (2007). Populist Radical Right Parties in Europe. New York:
Cambridge University Press.
Muller, Jan-Werner. (2016). What is Populism? PA: University of Pennsylvania
Press.
Netjes, C., and H. A. Binnema. (2007) ‘The salience of the European
integration issue: Three data sources compared.’ Electoral Studies
26(1): 39–49.
Norris, Pippa and Ronald Inglehart. (2019). Cultural Backlash. New York:
Cambridge University Press.
Norris, Pippa. (2019). ‘Varieties of populist parties.’ Philosophy & Social
Criticism 45(9-10): 981-1012.
Norris, Pippa. 2020. Global Party Survey dataset.
https://dataverse.harvard.edu/dataverse/GlobalPartySurvey
Pirro, Andrea. (2015). The Populist Radical Right in Central and Eastern Europe:
Ideology, Impact, and Electoral Performance. London: Routledge.
Polk, Jonathan, et al. (2017). "Explaining the salience of anti-elitism and
reducing political corruption for political parties in Europe with the
2014 Chapel Hill Expert Survey data," Research & Politics (January-
March): 1-9.
Poole, Keith T. and H. Rosenthal. (2001) ‘D-Nominate after 10 years: a
comparative update to Congress: a political-economic history of roll
call voting.’ Legislative Studies Q. 26(1), 5–29
Ray, Leonard. (1999). ‘Measuring party orientations towards European
integration: Results from an expert survey.’ European Journal of Political
Research, 36, 283-306.
Rooduijn, M., Van Kessel, S., Froio, C., Pirro, A., De Lange, S.,
Halikiopoulou, D., Lewis, P., Mudde, C. & Taggart, P. (2019). The
PopuList: An Overview of Populist, Far Right, Far Left and Eurosceptic
Parties in Europe. http://www.popu-list.org
Rooduijn, M, and T. Pauwels. (2011) ‘Measuring populism: Comparing two
methods of content analysis.’ West European Politics 34(6): 1272–1283.
Scarrow, Susan E., Paul D. Webb and Thomas Poguntke. (2017). Organizing
Political Parties Representation, Participation, and Power. Oxford: Oxford
University Press.
Simmons, Katie, Laura Silver, Courtney Johnson and Richard Wilke. (2018).
’In Western Europe, Populist Parties tap Anti-establishment
frustration but have little appeal across ideological divide.’
Washington DC: Pew Research Center.
Steenbergen, Marco R. and Gary Marks (2007) ‘Evaluating Expert
Judgements’, European Journal of Political Research 46(3): 347-66.
Torre, Carlos de la, and Cynthia Arnson. (2013). Latin American Populism in the
Twenty-First Century. Baltimore: Johns Hopkins University Press.
38
Measuring Populism Worldwide: Norris 1/8/20 8:50 PM
van Haute, Emily and Anika Gauja. (2015) Party Members and Activists.
Abingdon: Routledge.
Ware, Alan. (1996). Political Parties and Party Systems. Oxford: Oxford
University Press.
Whitefield, Stephen, Vachudova, Milada Anna, Steenbergen, Marco R.,
Rohrschneider, Robert, Marks, Gary, Loveless, Matthew P. and
Hooghe, Liesbet. (2007). ‘Do expert surveys produce consistent
estimates of party stances on European integration? Comparing
expert surveys in the difficult case of Central and Eastern Europe.’
Electoral Studies 26(1): 50-61.
Wiesehomier, Nina. (2019). ‘Expert surveys.’ In Kirk Hawkins, Ryan E.
Carlin, Levente Littvay and Cristóbal Rovira Kaltwasser. Eds. 2019.
The ideational approach to populism: concept, theory, and analysis. New York:
Routledge.
Wodak, Ruth, Majid KhosraviNik and Brigitte Mral. Eds. (2013). Right-Wing
Populism in Europe. London: Bloomsbury Academic.
Wodak, Ruth. (2015.) The Politics of Fear: What Right-wing Populist Discourses
Mean. London: Sage.
39