Economic Impact: Be cautious in reading too much into the data from political pollsters

Posted on November 7, 2016 by Chris Chmura

Not only will Tuesday’s election give us a new president, it also will tell us which, if any, pollsters were correct in predicting the winner.

What most people don’t realize is that the results of political polls vary greatly depending on the assumptions the pollsters make.

Ideally, poll respondents reflect the demographic makeup of the population. That almost never happens. So pollsters should weigh the sample results of their survey to reflect reality.

For example, 16.3 percent of the national population said they were Hispanic or Latino, according to the 2010 census. If only 9 percent of the people who answer the survey are Hispanic or Latino, then the pollster gives their responses a higher weight to determine the predicted winner.

Some other demographics that are typically included are age and gender. Other demographics that are more debated in terms of the ability to predict the next president are party registration and educational attainment.

Polls are typically conducted at the state level and then combined for national results. Polls conducted at a sub-state level may provide different results than those conducted at a state level. And, of course, getting statistically significant results becomes very expensive as more regions are polled.

There are two general approaches to the second consideration of who will show up to vote.

The pollster can ask the person answering the survey about their likelihood to vote. Or the pollster can use historical voter turnout on the assumption that past behavior is a good predictor of the future.

With all the data from the survey gathered, the weighting decisions of the pollster can have a large influence on the predicted election results.

Do we assume the same percentage of African-Americans will vote in 2016 as they did in 2012 or 2008? Will female and Hispanic turnout be higher? How does the likelihood of a person voting change based on two unpopular candidates? And how will a third candidate impact the overall results?

Based on the assumptions that are made, well-regarded pollsters are predicting different outcomes to the presidential election. On top of the question of whether we trust the pollsters’ assumptions is the 3.1 percent margin of error that is typically associated with surveying about 1,000 people.

As Election Day progresses, exit polls will provide better predictions of the announcement that will come later in the evening.

In the meantime, caution is warranted in reading too much into poll headlines.

This blog reflects Chmura staff assessments and opinions with the information available at the time the blog was written.