"Long Data" vs. "Big Data" in Elections

[Image courtesy of Flickr/njwilson23]

Recently, I wrote about “big data” – analysis of massive amounts of information that yields clues about individual behavior and preferences – and its potential value in the field of election administration.

Yesterday, WIRED Magazine had a fascinating article that suggests that we need to ” to stop getting stuck only on big data and start thinking about long data.” [emphasis in original]

The case for “long data” is based on the need for context:

[As] beautiful as a snapshot is, how much richer is a moving picture, one that allows us to see how processes and interactions unfold over time?

We’re a species that evolves over ages — not just short hype cycles — so we can’t ignore datasets of long timescale. They offer us much more information than traditional datasets of big data that only span several years or even shorter time periods.

This context is especially important given the simple human tendency to lose sight of longer-term change:

Datasets of long timescales not only help us understand how the world is changing, but how we, as humans, are changing it — without this awareness, we fall victim to shifting baseline syndrome. This is the tendency to shift our “baseline,” or what is considered “normal” …

Mind you, this idea isn’t a perfect fit for elections: the article defines “long data” as “datasets that have massive historical sweep — taking you from the dawn of civilization to the present day” – which isn’t really possible for an election system that has existed for “only” 220-plus years. But the idea that we need to put data into a historical context is crucially important and one which is often lacking in the cycle-to-cycle fights (what the author calls “hype cycles”) we have about election policy:

Big data puts slices of knowledge in context. But to really understand the big picture, we need to place a phenomenon in its longer, more historical context.

Of course, all of this pre-supposes data that, for the most part, simply doesn’t (yet) exist in the field of election administration. To that end, whether seeking big sets or taking the long view, the field of elections needs to renew and ramp up its commitment to data collection as a precursor to the kind of analysis that is increasingly becoming commonplace in other fields.