Political Polling Validity Becomes Shaky

Political polling is getting less reliable in predicting actual election outcomes. Reasons include the growing use of cell phones, reluctance to participate in telephone surveys and the rising cost of representative research samples.

Political polling is getting less reliable in predicting actual election outcomes. Reasons include the growing use of cell phones, reluctance to participate in telephone surveys and the rising cost of representative research samples.

Political polling doesn't seem to be as spot on as it used to be. Greater use of cell phones, wariness to participate in surveys and unrepresentative samples are among the reasons that political polls and election results turn out differently.

Cliff Zukin, a Rutgers political science professor and past president of the American Association for Public Opinion Research, writes in the New York Times that "polls and pollsters are going to be less reliable," so voters and the news media should beware.

"We are less rue how to conduct good survey research now than we were four years ago, and much less than eight years ago," says Zukin. "Don't look for too much help in what the polling aggregation sites may be offering. They, too, have been falling further off the track of late. It's not their fault. They are only as good as the raw material they have to work with."

Polling failures have been exposed in the most undetected 2014 mid-term election sweep in which Republicans captures both houses of Congress, Prime Minister Benjamin Netanyahu's solid victory in Israel and British Prime Minister David Cameron's relatively easy re-election win.

Cell phones are everywhere and increasingly have replaced landline telephones. Pollsters can find cell phone numbers, but federal law prevents calling them with automatic dialers. According to Zukin, "To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to working telephone numbers." That adds budget-busting cost to telephone surveys, which in turn lead to "compromises in sampling and interviewing."

Response rates to surveys have declined precipitously. In the 1970s, Zukin says an 80 percent response rate was considered acceptable. Now response rates have dipped below 10 percent. It is hard to draw a representative sample when large chunks of the population refuse to participate. Some cohorts, such as lower income household members, are more unlikely to participate than others, which can skew results. And it takes more calls to achieve a representative sample, which encourages corner-skipping.

Internet polling has emerged as a strong alternative. It is cheaper than telephone surveys and, at least or the moment, people seem more willing to participate, in part because they have more choice in when and how to respond.

But Internet use has built-in biases, too, Zukin notes. While 97 percent of people between the ages of 18 and 29 use the Internet, 40 percent of adults older than age 65 don't. "Almost all online election polling is done with non probability samples," Zukin says, which makes it impossible to calculate a margin of error. 

The most vexing polling problem is not a new one – determining who will actually vote. Public opinion polling is one thing; trying to predict the outcome of an actual election is another. Pollsters recognize that respondents will overstate their likelihood of actually voting, but have limited ability to identify who will and who won't cast ballots.

Non voting can occur for a mix of reasons – bad weather, lack of interest or political protest. Some registered voters simply forget to vote, especially in non-presidential elections. Less motivated voters vote in top-line races and leave the rest of their ballots blank, making it hard to predict the "turnout" for so-called down-ballot candidates and ballot measures.

Scott Keeter, who directs survey research at Pew Research, says the combination of these factors is shifting political polling "from science to art."

Political polls will continue to be magnets for media coverage, but readers should be aware that the results may not have as much validity as polling in the past.