"Formalized Curiosity": Orange County, CA Uses Regular Surveys to Improve Elections

[Image courtesy of wikimedia]

Two weeks ago, I wrote about Seminole County, FL’s voter experience surveys and their value to election officials and voters alike. This week in electionlineWeekly, Mindy Moretti heads (figuratively) all the way across the country to report on another well-developed survey program in California’s Orange County:

Research is formalized curiosity. It is poking and prying with a purpose — Zora Neale Hurston

Orange County, California Registrar of Voters Neal Kelley [who also serves as President of the California Association of County Election Officials] was used to hearing lots of great anecdotal evidence about the elections process in his county, but he was always curious about the hard facts of the process so in 2009 he decided to formalize that curiosity with a series of elections surveys.

“…[E]verything we were hearing from poll workers and/or voters was anecdotal – so I wanted some concrete ways to measure results and provide a solid base to capture some ways to monitor metrics,” said Kelley.

The county has been conducting Election Day surveys for more than five years now and runs the program for all elections — including special elections. The county released its 120-page report on the June 3, 2014 primary in September and the November 2014 report will be available shortly.

“When we first started we quickly identified areas where we could improve immediately and were able to pivot pretty quickly between the first two elections where we began conducting the surveys,” Kelley said.

Kelley said the early results did surprise him a bit. There were areas where he felt his office was doing well — poll worker training, poll worker materials, voter phone wait times, etc. but the survey results told him otherwise.

“It would be easy to sit back and say nothing surprised me but that’s not true – we all need to be self effacing and prepared to confront areas where we can improve – and we have done that as we move forward, the survey results show this,” Kelley said.

The office also monitors the data during weekly planning meetings and management meetings to make adjustments on the fly – not just post election. That, Kelley said, is where the real time data becomes so valuable.

Currently there are 11 different surveys that are distributed:

• + Poll Worker Survey asks poll workers to assess the various components of their volunteer experience;
• + Training Survey is distributed at the end of election night and pertains to how well the office prepared poll workers for Election Day;
• + Delivery Survey asks polling place hosts to assess the delivery company that was tasked with delivering election supplies and equipment to their location;
• + Polling Place Survey asks polling place hosts about their experiences receiving, storing and returning equipment and supplies;
• + Election Supply Distribution Survey is provided to inspectors when they picked up their precinct-specific supplies for Election Day;
• + Phone Bank Survey is taken by members of the public who call the Public Phone Bank and poll workers who call the Poll Worker Phone Bank;
• + Coordinator Survey is distributed to the coordinators in order to rate their experiences leading up to and on Election Day;
• + A-Team Member Survey is provided to A-Team members (back-up poll workers) as they are deployed to a polling place on the morning of the election;
• + Recruitment Survey was developed and implemented in 2010 as a means to measure the level of customer service provided by staff members who actively recruit volunteers;
• + Collection Center Survey was utilized for the first time in June 2014 and asks Collection Center Workers for their feedback on the quality of training and preparation; and
• + Candidate Filing Survey is provided to candidates who completed filing in the registrar’s office or online.

Kelley said they didn’t have target number of surveys; 11 was the number of areas his office felt that needed to be measured.

Since 2009, using the data from the surveys the county has retooled its polling place manual from start to finish, completely redesigned the provisional process and envelopes, adjusted resources on phone banks, improved training materials and changed the way it packages elections supplies, to name a few.

“I could go on – the survey program has made us a much better operation,” Kelley said.

The office also surveys voters by capturing data on any voter that interfaces with the office in the last 120 days prior to an election, including Election Day. What they don’t have at the moment is a polling place survey for voters, but Kelley said they are working on that.

Kelley said he hopes to have the polling place survey as well as a survey for vote-by-mail voters in place by 2016.

The surveys are a team effort among the divisions in the department, Kelley explained.

“Each was tasked with identifying the questions that would be needed to monitor the key metrics — customer service, equipment delivery, service times, etc.,” Kelley said. “We have not worked with consultants – we have some very bright folks on staff who have statistics backgrounds, which has certainly helped.”

The cost of conducting and producing the survey is built into the office’s budget. Internal staff members are used to prepare, publish, collect and monitor the data. The county auditor is occasionally used to verify the data since it is an internal program.

Kelley said he believes conducting election surveys is something every government organization can afford and benefit from no matter the size of the agency because of the importance of listening to stakeholders.

“…[E]ven basic survey collection tools can help,” Kelley said. “And there are so many free data collection tools that can be used online. We offer printed and online options, but simple online tools are available – they don’t have to be coded from the ground up.”

Thanks as always to Mindy for her reporting on this issue – and thanks especially to Neal and his team for an excellent example of using simple survey research to drive continuous improvement in election operations.