MIT's Charles Stewart on New Pew Elections Performance Index

[Image via MIT]

Last week, The Pew Charitable Trusts released the latest edition of its Elections Performance Index (EPI) with data from the 2014 midterm election. In the most recent edition of electionlineWeekly, MIT’s Charles Stewart – a prime force behind the EPI – talked about what it means for American election administration now and going forward:

On [August 9], The Pew Charitable Trusts released the latest version of the Elections Performance Index (EPI), its effort to take ideas proposed by Heather Gerken in The Democracy Index and turn them into flesh and blood (or at least electrons).

The website captures what happened during the 2014 midterm election, adding to existing measures from 2008, 2010, and 2012, as well. Having data from a series of elections makes it possible to examine the process of change across time. Most importantly, now that the EPI has two midterm elections under its belt, it is possible to do an apples-to-apples comparison of each state with how it performed in successive midterm elections.

The headline for this release — that the administration of elections in the U.S. continues to improve, slowly but surely — will certainly strike a discordant tone with many in the public, who have been fed a steady diet of stories claiming that American elections are rigged or vulnerable to hacking. Yet, the EPI points to a set of deeper truths about American elections that, one hopes, will gain the attention of the public, lawmakers, and election administrators once this election season is over. [emphasis added]

The EPI is constructed by combining 17 measures of election administration, most of which are performance outputs, such as the percentage of absentee ballots rejected and the percentage of UOCAVA ballots unreturned. As explained in the methodology document that accompanies the EPI website, these 17 measures were chosen because they provide a comprehensive view of election administration at the state level, conceived along two dimensions.

Along the first dimension are the functional requirements for potential voters to have their ballots successfully counted: they must be registered, successfully cast a ballot, and the ballot must be accurately counted. Along the second dimension are the two normative goals we wish to achieve through our electoral process: it should be convenient to vote and the electoral process should be secure.

Conceived of this way, it is important that the EPI indicators reflect the nitty gritty of election administration. For the most part, a state can improve its performance by reforming or tightening up its administrative practices. For instance, a state that integrates its voter registration system with its Department of Motor Vehicles (DMV) through online voter registration, will improve along the dimension that penalizes a state for rejected registrations and more provisional ballots.

Drilling down into the index, there are two indicators that provide the biggest boost to overall scores in 2014, which increased 5.1 points on average. The first is the so-called “data completeness” indicator, which gauges how completely a state provides core data to the Election Assistance Commission’s Election Administration and Voting Survey (EAVS). Readers of this posting likely know that the EAVS is a large, somewhat unruly instrument that states and localities have struggled to complete. The EPI doesn’t expect states to complete the survey entirely, but rather focuses on a small number of core measures that seem fundamental to gauging a state’s electoral workload. It seems reasonable, in other words, for a state to be expected to report how many new voter registrations were processed, how many provisional ballots were cast, and how many absentee ballots were returned for counting.

In 2010, overall data completeness averaged 94 percent, which means that the average state reported 94 percent of this core workload data. In 2014, overall data completeness grew to 97 percent. Expressed another way, in 2010, only 12 states reported all the core EAVS data; in 2014, 27 states did.

Data completeness is not a sexy measure, but touches on the core of managing with metrics — without metrics, it is impossible to manage using them.

Another pair of indicators that saw significant improvement pertained to military and overseas (i.e., UOCAVA) ballots. From 2010 to 2014, the average rejection rate of UOCAVA ballots fell from 6.6 percent to 4.8 percent, and the percentage of UOCAVA ballots that failed to be returned fell from 53 percent to 43 percent.

In the course of developing the EPI, a topic the advisory committee continually wrestled with was how to account for factors that are outside the control of election administrators, at least in the short term. A good example is turnout, especially in midterm years. The advisory committee eventually came to the conclusion that turnout was an indispensable component of any index that tried to assess the performance of an electoral system. What this means for 2014 is that virtually all states suffered a little bit in their EPI scores because average turnout fell from 44 percent in 2010 to 40 percent in 2014. (Thirty-nine states saw turnout declines from 2010 to 2014.)

The states that saw the biggest turnout declines from 2010 to 2014 were those that had a U.S. senator on the ballot in 2010 but not in 2014. (For Senate geeks, these are the 13 states whose senators are in classes I and III.) Among these states, turnout fell an average of 6.8 percentage points. This compares with an average drop in turnout of 2.5 points among all other states. Certainly, these differences in turnout declines were not because of differences in how elections were run.

It should be noted that consumers of the EPI don’t have to take the judgement of the advisory committee as the last word on a topic like this. If you don’t think turnout should be used as part of the EPI — either as a general matter, or in midterm elections — the EPI website allows you to remove turnout from the index rankings with the click of a mouse. (Spoiler alert: When you do this, very few states move very far in their ranking. Only three states move more than five rank positions when turnout is excluded.)

Because the EPI was inspired by Gerken’s Democracy Index idea, there is no hiding the fact that one goal of the EPI is to instill a little bit of competition among states so that all will continue improving how they run their elections. There’s another goal that’s equally important. It is hoped that the EPI will provide a starting point for conversations at all levels of government about what makes for a well-run election, and how to achieve it.

Let the conversation begin.

This is an extremely important conversation to have – especially, as Charles notes, given the rising chorus of complaints about election administration that always seem to accompany a national campaign. Pew’s EPI is an incredibly valuable tool for assessing the health of American democracy; thanks to the EPI team and advisory board for its hard work, and thanks especially to Charles Stewart for being a champion for evidence-based analysis of elections at a time when it is sorely needed.

This is a crucial conversation – stay tuned …