Elections, Data vs. Metrics – and Ted Williams

[Image courtesy of foxnews]

Earlier this week, the Harvard Business Review ran an interesting piece by two non-profit leaders looking at how (and how not) to use data in support of a larger enterprise. Here’s how it opens:

How many views make a YouTube video a success? How about 1.5 million? That’s how many views a video our organization, DoSomething.org, posted in 2011 got. It featured some well-known YouTube celebrities, who asked young people to donate their used sports equipment to youth in need. It was twice as popular as any video Dosomething.org had posted to date. Success! Then came the data report: only eight viewers had signed up to donate equipment, and zero actually donated.

Zero donations. From 1.5 million views. Suddenly, it was clear that for DoSomething.org, views did not equal success. In terms of donations, the video was a complete failure.

What happened? We were concerned with the wrong metric. A metric contains a single type of data, e.g., video views or equipment donations. A successful organization can only measure so many things well and what it measures ties to its definition of success. For DoSomething.org, that’s social change. In the case above, success meant donations, not video views. As we learned, there is a difference between numbers and numbers that matter. This is what separates data from metrics.

It was hard for me to read that and not think about the current efforts to improve the election process. In one way, now is a really exciting time in the field; you’re starting to see election officials (old and new) thinking about how better to capture and use data in their work. The danger, of course, is that focusing on the wrong thing – or even too much on too narrow a sliver of the right thing – can distort your performance.

That’s why I hope the current attention to line length and wait time doesn’t become a primary metric that crowds out or de-emphasizes other data that measure concepts like accuracy or cost. If that happens, we set up incentives for election officials to prioritize one aspect of elections (wait time) over everything else.

The downside of a misguided or over-emphasized metric is that it can result in individuals or organizations choosing to overlook actions or approaches that might be more valuable in the longer run.

And that’s where Ted Williams comes in. Williams is widely regarded as one of the best hitters in major league baseball history, and was legendary for his ability to see and identify pitches almost as soon as they left the pitcher’s hand, making him a selective hitter willing to take walks to get on base – a skill highly prized in today’s game. But in Williams’ day, the game favored hits, not walks, as Grantland’s Katie Baker heard at the recent MIT Sloan Sports Analytics Conference:

In a panel I attended on “Hall of Fame Analytics,” which focused on the way HOF voters make their decisions, ESPN’s Buster Olney made an interesting point that goes the other way. The analytical types, he said, often deride older-generation players who may look good by [traditional hitting] standards like RBIs but don’t hold up to the newer Moneyball-style rubrics [like walks and on-base percentage]. But, he argued, those guys weren’t trying to do the sorts of things that the sabermetrics crowd approves of, such as take walks. [Footnote: When Ted Williams took walks, Olney said, he was considered a selfish player.] For years, MLB salaries were strongly tied to RBIs, Olney found when he went back and looked. How are we to know that they wouldn’t have been just as good by more modern metrics if that’s what they were getting paid to do? As is usually the case, one side of the story leaves you half-blind.

The important thing to remember as the use of data expands in elections will be to choose data – and metrics – that don’t leave us half-blind.