[Image via courtoconnell]

Katy Owens Hubler of Democracy Research LLC has a really interesting and informative piece in the current electionlineWeekly discussing a topic you may have heard about but didn’t realize was so important – a common data format for elections:

If you’ve been to national election conferences in the last several years you’ve likely heard the term “common data format” or CDF. What is a common data format and why is it important for elections?

What is a common data format?
In this day and age, data is king (or queen). As with everything else, elections administration has become more data heavy and more device heavy.

It wasn’t always this way. Think back to the analog age of giant cumbersome lever voting machines when there was just one “voting machine.” It was mechanical, not electronic, and wasn’t attached to any other machines that helped it serve its purpose. It allowed electors to cast their vote, and it tabulated the votes without the need to “talk” to anything else.

Now, election systems are much more than a big bulky lever machine. There is a web of different software and hardware systems that need to interact with each other in some way. Data is at the core of all of these systems, from the databases that are used to maintain voter lists, to electronic poll books used to check voters in, to devices used to tabulated votes, to election night reporting systems that display results.

Sometimes these systems can reflect a United Nations without a translation service. Different devices made by different manufacturers and for different purposes all speak different languages. It can take quite a bit of work to translate the data coming out of one device into a language that can be imported into another device. This could mean that software is needed to reformat the file in the middle, or even that individuals are “keying-in” data from one data set into another manually.

A common data format (CDF) allows each of these systems to “speak the same language.” Having data in the same format across all systems allows for more streamlined exports and imports from one system to another. A CDF advances the goal of interoperable election systems – systems that work together seamlessly no matter who the manufacturer is.

What are some advantages of a CDF?

  • Increased efficiency. If different systems “talk,” then officials don’t need to re-key the same data across different databases, which often introduces errors. Common formats streamline imports and exports between different systems.
  • More flexibility. Eventually common formats will allow for easier integration of new components, components from different vendors, and commercial off-the-shelf (COTS) components. Election officials would have more choices in what is available for purchase, which in the long run could save money.
  • Enhanced security through transparency. If data is available in a common format, rather than a proprietary format, it increases transparency and makes the data easier to analyze for a greater number of citizens and groups. More eyes on the data makes it easier to test for accuracy and determine whether the data has been changed once it’s been exported.
  • Easier to test. Devices that use a CDF will be easier to test, potentially speeding up the testing and certification process.
  • Integration with other data. Once data is in a common format, it’s also simpler to provide data for other needs, such as the U.S. Election Assistance Commission’s Election Administration and Voting Survey (EAVS), the Voting Information Project (VIP) and other requests from the media, legislators, campaigns, etc.

If election data is easier to consume, more organizations want to use it and present it in cool ways – everyone from Google to MTV can create maps and graphics for presenting voter information and election results.

What is available now and how can I use it?
CDFs for various aspects of the election process are being developed by the National Institute of Standards and Technology (NIST), in collaboration with the EAC and with the help of public working groups.

The working groups consist of state and local election officials and their staff, voting equipment manufacturers, and election experts from all over the country. Anyone can get involved – see NIST’S Interoperability Public Working Group page for information and for joining.

The Election Results Reporting specification (see NIST SP 1500-100 Version 1.0) is the most developed of the CDF work. If you are a state or local election official who has been frustrated by the process of compiling and reporting results with candidates and the media breathing down your neck, this is something you may want to review.

Without a CDF, local jurisdictions have to export results from their systems, translate it in some way (using software or perhaps manually) or even type in result information by hand. Having a CDF allows result information to be automatically downloaded from local jurisdictions and uploaded into the statewide reporting system. It reduces errors from manual entry, which means that candidates, the media and the public get to see results faster and in a more uniform way.

A CDF also permits states and jurisdictions to form partnerships with other organizations. Using the NIST SP 1500-100 specification permitted the Virginia Department of Elections to work with Google to present results of the 2017 general election (which included a high profile gubernatorial race). Virginia published result information in the 1500-100 format, which allowed Google to directly consume the data and display results on their search page.

One advantage that Virginia saw in forming this partnership was the amount of web traffic absorbed by Google, reducing direct web hits on the Department of Elections page. Reducing the traffic load on the state’s site can save money on web hosting, and also decrease the risk of the website failing on election night.

Matthew Davis, the Virginia Department of Elections Chief Information Officer, notes that on election night, “The more places you can have official access to the data, the better.”

Official data is getting out to a larger audience, and is available through other sources in case of a server outage at the state election office.

Virginia also discovered an unexpected benefit of making changes to their internal data infrastructure to accommodate 1500-100 – it was also compatible with the Voting Information Project (VIP) specification, permitting the state to implement VIP feeds much more quickly.

VIP works with 46 states and the District of Columbia, thus many are already aware of the advantages of the partnering with the project to provide official information to voters, such as what is on their ballots and where to vote.

Maria Bianchi, the VIP Project Manager at Democracy Works summarized the advantage for states in using the specification: “VIP data comes directly from the states, so the information is up-to-date and accurate. Technology companies and civic engagement groups are then able to innovate and build effective voter outreach campaigns around the official data that the states provide.”

The CDF for election results reporting was the first to be completed and is therefore the most used at the moment, but there are other ways that states can use the CDFs still in development. For example, states looking to replace aging voter registration databases can consider adding language about CDFs into the development of requirements for the new system, a request for proposals (RFP), or when contracting with a new vendor. The Voter Records Interchange (VRI) CDF is near completion and deals with data interchange from online voter registration systems, voter registration systems, third-party systems, and motor vehicle agencies.

The VRI CDF also has potential to help election officials consume and integrate data from the department of motor vehicles and other registration agencies more easily. This can be helpful for any state, but especially so for those looking at a more automated voter registration system and list maintenance procedures.

The next version of the EAC’s Voluntary Voting System Guidelines (VVSG) will require the common data format. Election officials can help with feedback on these draft requirements and start requiring CDFs in new election products to get ahead of the curve.

To make sure each CDF is developed accurately and reflects the real-life experience of election offices, we need jurisdictions that are willing to try them out and give feedback. If your state is looking at implementing an updated voter registration database, for example, or if you’re looking to get proposals for new election system components, get in touch about how you might be able to use the CDF work.

See the sign-up information at http://vote.nist.gov/interoperability or contact the Interoperability Working Group Chair John Wack for more information.

Thanks to Katy (who I first met when she was at NCSL, but who has stayed in the business and is one of our most valuable #electiongeeks) for this primer – and thanks as always to electionline’s Mindy Moretti for “commissioning” and running this piece. Happy spring – and stay tuned …

Posted in Uncategorized and tagged