Can Polls be Trusted?

September 16, 2020

This week we begin with a piece focused on polling, a useful and particularly timely proxy for all market research. With 48 days left until the conclusion of the 2020 U.S. Presidential Election, we’re looking at one poll per day as pundits try to hone their predictions for who will take the White House. Four years ago few, if any, analysts accurately predicted the outcome of the election. This time around let’s better understand what’s happening, and why.  

Perhaps the political question I have received the most in the past four years is what happened to the polls in 2016 – why weren’t they accurate, and what do I need to understand about the polls this time around?  As a lifelong pollster and strategist, I know all too well that polls can inspire winning strategies – but used incorrectly, they can also steer predictions dangerously off course.  

First, understand that polls are but a single frame in a long movie. And try as you might, you’ll be hard pressed to guess the plot of a movie from a single still image. Yet that’s what people insist on doing with polls. To derive real value from polling, you need to assess the trends that develop over time, particularly as those trends accelerate in the run-up to Election Day, ultimately helping forecast how the chips will fall on November 3.  

Second, contrary to popular belief, the national polls that were fielded in the last presidential election were not inaccurate; the failing was in the analysis that accompanied them. The polls demonstrated that while Donald Trump was faring poorly in states that typical Republican candidates tend to do well in, he had amassed support among Midwestern voters, and so Hillary Clinton’s leads in those polls were not as large as assumed when the Electoral College began to take shape. Some of the state-by-state polls, on the other hand, were wrong; polls in Ohio and Iowa picked up the Trump surge, but those in neighboring states showed solidly blue support, as usual. The Democratic Super PAC clearly saw the disconnect, rushing $6 million into all the critical swing states in the last week of the election, but – as time would show – by then, they were too late. 

Third, fast forward to our current election and the chatter about a cohort of “shy voters” who may swing the election in Trump’s favor come Nov. 2. I wrote about silent conservatives in my book Microtrends, observing that while there do exist voters who are reticent to voice their political support, the overall effect they have on the election is not very large – maybe one or two points. A recent study by data collection company CloudResearch has revived the discussion, showing that conservatives were twice as likely to obscure their candidate preference in telephone polls than liberals.  

I personally saw this in New Hampshire in 2008 when Hillary cinched the primary there despite few voters in any poll being willing to say they were voting for her in the last weekend of surveys. And I have seen it elsewhere, in countries where freedom of speech was constrained and people were afraid to voice their real views for fear of being arrested. Freedom of thought and expression are critical for polls to be accurate, and the current level of polarization in the United States makes expression of political views difficult for some voters. So, ask yourself, “do Americans feel free to express their political views openly?” If you think not, then add a shy voter factor to your polling analysis.  

Finally, as you read polls, look carefully at the questions being posed to the voter — are they one-sided or balanced? Perhaps my largest gripe with most polls today – and a source of error in understanding them – is that the questions themselves are often leading questions that reek of bias and establish false-choices for the respondent. Ask just one question and you are polling to fill in a narrative; ask about both and you are really doing what polls do best — finding how and where voters hold seemingly contradictory views. For example, in June, people voiced support for both the police and the Black Lives Matter movement. They viewed the choice between social justice and effective policing a false choice — they want both.  

We wouldn’t have a relatively close election if there were not two sides to every voter issue, whether you agree with the other side or not. It’s important that the polls reveal not only what voting choices are out there, but illustrate why people are making those choices. Increasingly, polls do a poor job of the latter by asking selective questions to confirm a narrative.  Another problem is that swing voters are typically cross-pressured – they want both better policing and an end to racism. That duality is exactly what makes them a swing voter. If you miss that, you miss the whole story of the election.  

So what is the story of this election? 

As I wrote in The Hill yesterday morning, here are the seven critical factors shaping people’s votes as we go into the chute (the period from Labor Day to Election Day)  and the three in particular that may very well decide the outcome.