Want Accurate Predictions Again? Pollsters Say Trump’s Exit Can Do That

(CN) — The polls were wrong. Again.

Joe Biden was up by 10 points or more in surveys taken just before the election by NBC News, the Wall Street Journal, the Economist, CNBC and Quinnipiac. Once ballots were tallied, however, his 3-point lead over Donald Trump in the popular vote is barely better than that of Hillary Clinton in 2016.

The RealClear Politics polling average had Biden winning Florida by about 1 point, but he lost by 3.3. Wisconsin polls had Biden up by 6.7 points; he won by only 0.7. The last five polls in Texas showed Trump winning by about a point, but he won by 6.

And it wasn’t just the presidential race where the polling consistently overestimated Democratic strength. Polls in Senate races were even further off, with surveys in nearly every marquee contest showing Democrats doing far better than they actually did.

In Maine, some 14 polls were taken this year and every single one showed Republican Susan Collins losing. The last three polls before the election showed her behind by 4, 6 and 7 points. She won by almost 9.

In Montana, the last four polls before the election showed Republican Steve Daines winning by about a point. He won by 10.

In South Carolina, three of the six polls taken closest to the election showed the race as a dead heat. Republican Lindsey Graham won by well over 10 points.

And there’s more. Polls significantly overestimated the Democratic vote in key Senate races in Georgia, Michigan, Iowa and North Carolina.

Coupled with the polls’ well-known failure in 2016 to predict Trump’s victory and Republican success in the Senate, the latest whiffs are a black eye to the industry.

“The biggest problem in 2016 and 2020 was not that the polls were far off; it’s that they were all off in same direction,” said Andrew Smith, director of the University of New Hampshire Survey Center.

“If the problem were a random error, it would have gone both ways,” he said. “Some mistakes would favor Democrats and some would favor Republicans.” But that didn’t happen.

So what’s causing the systemic distortion? Polling experts say there’s a simple two-word answer: Donald Trump.

In the 2018 midterms, when Trump’s name wasn’t on the ballot, the polls if anything were more accurate than usual, said Terry Madonna, director of the Franklin & Marshall College Poll. It’s only when Trump is running that the polls fail to pick up Republican strength.

There has been a lot of speculation as to why this is, ranging from “shy” Trump voters who are reluctant to admit their preferences to deliberate media bias.

Pollsters have other explanations. And interestingly, they say the problem in 2020 was different from the problem in 2016.

The polling industry conducted extensive postmortems to figure out what went wrong in 2016, Madonna said. Franklin & Marshall “conducted exit polls of people we polled earlier to see what happened.”

Two things happened. One of them was that undecided voters in 2016 broke extremely heavily for Trump in the last days of the election, after FBI Director James Comey reopened his investigation of Hillary Clinton’s emails.

But there were few public surveys in the critical battleground states in the week before the election and thus no one picked up on the shift.

Franklin & Marshall’s postmortem revealed that “a huge percentage of people made up their minds or changed their minds in part due to Comey,” Madonna said.

The other problem was that people without a college degree voted in unusually large numbers and voted heavily in favor of Trump.

As a consequence, many polls have begun “weighting” their results for educational level so that mistake doesn’t happen again.

Smith said his team has studied weighting for education in other races and found that it makes no difference in races where Trump isn’t on the ballot.

“2016 was unique,” he said. “Before 2016 it didn’t have any impact; then suddenly it did.”

But all the efforts to fix the polls in 2020 didn’t work. “Weighting for education didn’t fix the problem because that wasn’t the problem” this year, Madonna said.

There have also been efforts to capture “shy” Trump voters by asking indirect questions (such as which candidate respondents think their neighbors are voting for), but that didn’t solve things either.

The real problem in 2020? Large numbers of Trump voters simply refused to participate in polls at all.

“There was a bias among Trump supporters not to participate,” said Smith.

In the past, Smith said, research has shown that the race and sex of the polling interviewer impacts the results since respondents sometimes tailor their answers to what they assume the interviewer wants to hear, which is known as social-desirability bias.

But now, according to Smith, the perceived political preferences of the research organizations conducting the polls — which are often media outlets — are skewing the results as well.

“Trump’s base of support base desperately dislikes the media, and that’s an understatement,” said Madonna. “They distrust the media, so they distrust the polls, because they associate them. And they refuse to talk to them.

“It’s not that there are a lot of ‘shy’ Trump voters who are afraid to state their preferences,” Madonna continued. “They’re not afraid. They just think that participating in polls supports the big media. It’s not social-desirability bias; it’s anti-media bias.

“And that carries over to the Senate races,” Madonna said, because the same group of voters are refusing to participate.

“It’s a selective nonresponse problem,” said Smith.

The big question for the industry is whether this problem is unique to Trump or whether his base of supporters will continue to boycott pollsters after Trump is gone from the scene. That’s not clear, although Madonna thinks the fact that the 2018 midterm polls were accurate suggests that the problem will likely go away once Trump does.

Some pollsters blame the media for overrelying on polls as “clickbait” and failing to educate voters about their purpose and limits.

“The margin of error matters,” said Madonna, and polls “are intended to be snapshots. They’re not predictive.” (Certainly the polls could not have predicted that Comey would reopen an FBI investigation.)

Making matters worse, said Smith, is that some polling aggregators such as FiveThirtyEight have begun converting polling data into probabilities of winning, which can be misleading.

“If you say that a candidate has a 70% chance of winning, a lot of people think that means that the candidate will definitely win or that they’ll win by 70% to 30%,” Smith said.

Also, polls vary a great deal by quality, ranging from live telephone interviews to online polls of randomly selected people to opt-in polls to automated “robopolls” directed only at landlines. Many polling aggregators don’t consider the quality of the polls, Smith said.

“The people behind them are statisticians, not survey researchers,” he explained. “You might think that using more polls is preferable, but not necessarily. Averaging good polls and bad polls is like mixing clean water and muddy water; it doesn’t make things better.”

Categories / Politics

Subscribe to Closing Arguments

Sign up for new weekly newsletter Closing Arguments to get the latest about ongoing trials, major litigation and hot cases and rulings in courthouses around the U.S. and the world.

Want Accurate Predictions Again? Pollsters Say Trump’s Exit Can Do That

Subscribe to Closing Arguments

Additional Reads