Home World Nate Cohn: Explains What the Polls Got Wrong, United States

Nate Cohn: Explains What the Polls Got Wrong, United States


Nate Cohn: Joe Biden was elected the 46th President of the United States last week, defeating Donald Trump by a margin of several percentage points across the country. The margins in key swing states, on the other hand, were closer than polling averages predicted. Though Biden won the three Midwest battlegrounds—Michigan, Wisconsin, and Pennsylvania—where Trump barely won in 2016, and the former Vice President is likely to win in previously red states like Georgia and Arizona, major networks took several days to call the election results, and what appeared to be a landslide for much of the campaign ended in yet another scramble for electors.

Nate Cohn Explains President of the United States Last Week

Nate Cohn
I recently chatted with Nate Cohn, a New York Times domestic correspondent who oversaw the newspaper’s polling this cycle. (It’s worth noting that Cohn and I worked together at The New Republic and are good friends.) We talked about how the pandemic affects polls, the significance of statistics in election coverage, and the Times’ contentious “election needle” during our chat, which has been edited for length and clarity.
What surprised you the most as the results came in, or as they’ve come in?

Hispanic voters. Trump’s popularity is surging in Hispanic communities around the country. It was hinted at in the polls leading up to the election. The President always polled better among nonwhite voters, especially Hispanics, than he did four years ago, but the scale of the movement went much beyond expectations. We discovered this late at night in Miami-Dade County, where no one predicted the President would perform as well as he did. And it appears to be true, to varied degrees, across the country among Latino voters, as far as I can discern. It’s a true down-ballot situation. It’s not like the President was the only one involved. And I believe that is a significant political tale.

The white, rural, Midwestern vote was the second thing that astonished me. According to pre-election polling, Joe Biden was faring better than Hillary Clinton four years ago among white voters without a college diploma. And those improvements were just not realised. Across most of rural America, the results resembled those of 2016, and there were several locations where Donald Trump scored better in white working-class neighbourhoods than he did in 2016.

I had problems getting specific data for what percentage of Hispanic voters Trump won in 2016 and what percentage he won this time because the definition of “Hispanic” tends to shift from survey to poll and exit polls are unreliable. What do you think the magnitude of that shift is across the country?

I believe it will be a double-digit swing in the President’s favour. I haven’t done any conclusive math on these numbers yet, and it’s still too early to do so. But, yeah, that would be my first instinct.

The most obvious cause for this, I believe, is our political system’s education divide, which is showing itself along racial lines. Do you have a counter-argument?

That, I believe, is correct. And I don’t believe this was an election about immigration. From Donald Trump’s announcement that he was running for President to his policy plan to build a wall, immigration was a prominent subject in the 2016 election. The campaign revolved around the President’s immigration policy and his attitude toward Mexican-Americans. And I’d argue that it hasn’t been a central topic of this presidency, and it won’t be a central theme of the 2020 campaign either. So it seems sense to me that if we stop talking about immigration and Hispanic voters begin to evaluate the President without that in mind, they will begin to move in ways that are comparable to demographically similar white voters four years later. That appears to be a possibility.

It would also imply that there were more Hispanic men than Hispanic women, right?

According to a lot of pre-election survey data, the Hispanic gender gap widened dramatically in this election.

It’s funny that you mentioned it wasn’t a major topic of his campaign or presidency. It seems to me that it was a major topic throughout this President’s term, but not so much during his reelection campaign.

What year would you pick if you had to pick a year during his presidency that was dominated by immigration?

I believe the year will be 2018. Separation of the family, followed by the caravan.

Yeah. That, I believe, is correct. I believe there was a time in late 2018 when immigration was front and centre in our politics. That was undoubtedly untrue of 2020, and I believe it was also untrue of much of the Trump Administration. And, in 2016, I’d say immigration was always near the top of the list.

Trump was polling ahead of Republican Senate Nate Cohn candidates in many pre-election polls. Republicans, on the other hand, appeared to fare better than Trump in the general election for the House of Representatives and in several of these Senate races. What made you think that?

I’m not sure why it happened to be the case. One of the most intriguing aspects of this year’s polling error is that it was greater down-ballot than it was at the top of the ticket, when in 2016, the opposite was true. So, while most of our explanation for what went wrong in 2016 revolved around looking for things that were primarily about the President, I’m not sure the pollsters are correct in assuming that this year’s polling inaccuracy is unique or exclusive to the President.

In 2016, nationwide polls were a couple of points wrong, while Midwestern surveys, particularly those that didn’t weight by education, were even further off. There were a lot of misses this year with your polls for the New York Times and other polling. And these were the polls that took education weighting seriously, made a lot of steps to correct problems after 2016, and, in your case, nailed the 2018 midterms. Do you recall what occurred this time?

 I have a few ideas for you | Nate Cohn

But, before I go there, let me state that I agree that this was a lot worse polling error than in 2016. In national polls, there was a greater polling miss. It was a bigger polling mistake for the industry’s most well-known and expensive polling firms. Even if, as you noted, many state pollsters took steps to increase the percentage of white voters without a degree in their surveys, the polling error will be just as bad. State polls are also looking a lot like they did last year.

However, if state polls are just as terrible as they were in 2016, despite efforts that we know boosted the President’s standing in the polls, we can argue with absolute certainty—and I know this was true in our data—that the underlying survey data must be worse than it was in 2016. If all pollsters used the same technique as in 2016, the polls would have been significantly worse this year than they were in 2016. And that’s quite intriguing. As I previously stated, I can provide you with a number of theories.

What theories do you have?

What has changed since 2016 is the key framing. What would cause the polls to be worse now than they were previously? So one hypothesis is that four more years of Trump, and that as American politics became increasingly defined along the lines of your opinions about the President, and as previous party allegiances faded, non-response bias in polling became more and more connected with Presidential vote choice.

Another view is that “the resistance” was the catalyst for the polls to collapse. Consider all of the left’s political activity, including the millions of dollars spent to support Jon Ossoff in 2017 and Jaime Harrison in 2020. This reflects a significant growth in progressive political participation. We know that voters who are politically active are more likely to respond to polls. As a result, it’s possible that, as the Trump presidency has energised the Democratic base, that same group of voters has increased their willingness to participate in political polls.

Another option is this year’s large turnout. Because we think of turnout as an additional component that polls must get right beyond just getting a nice sample of the population, we in the polling world have tended to presume that increased turnout makes polling easier. This year, though, there was a massive increase in turnout, which most people assumed was excellent for Joe Biden. Perhaps it was beneficial to Joe Biden. But I believe we must also be open to the possibility that it was not in Joe Biden’s best interests. I can tell you with certainty that the electorate in Florida, where we were gathering turnout data live on Election Day, was more Republican than it was in 2016, more than our polls predicted, regardless of your likely-voter methodology. That could be true in other parts of the country. I’m not sure. It’s possible it’s not. We just do not have that information at this time.

However, many recent surveys have shown that the margin between registered and likely voters has narrowed, and in some cases, it has reversed, with the Democrats doing better among both likely and registered voters. A few late examples: according to a CNN survey, Biden is up ten points in Pennsylvania, but just five points among registered voters. In Pennsylvania, I believe the final ABC News/Washington Post survey found Joe Biden to be doing better among likely voters than among registered voters.

Finally, I’d mention that it’s possible it was the coronavirus. You may recall that a year ago, we published a series of surveys showing Biden marginally ahead of Trump and Elizabeth Warren trailing Trump, and those polls were far more accurate than those conducted since then.

They were more accurate, in the sense that they were closer to the final Biden-Trump vote totals | Nate Cohn

They were, in fact, a lot closer to the final 2020 results than our final 2020 polls. And that interests me because our surveys in 2018 were also extremely accurate. The coronavirus is one thing that has transpired since then. And, when I first started looking at the election results on Tuesday night, one of the first things that struck me was Biden’s poor performance in the coronavirus hotspots. Remember how Biden was supposed to do exceptionally well in Wisconsin, thanks in part to the coronavirus? That was not the case.

Wisconsin was undoubtedly the swing state with the highest polling mistake, with Biden leading by a wide margin on election night.

Yeah. But, isn’t it true that part of the reason Biden was gaining ground in Wisconsin was because the coronavirus was ravaging the state? You may recall that over the summer, studies claimed that “when coronavirus strikes, Biden performs better.” What if, when the coronavirus strikes, Biden does better in polls but not in reality, since a large number of Democrats take the virus seriously and remain home, and Republicans do not?

I’m not saying that’s accurate, but we do know that after the coronavirus outbreak, survey response rates increased. There were numerous articles on the subject. And if we assume that the Democrats were the ones who, on balance, took the coronavirus more seriously, perhaps that explains it. It would also do a better job of clarifying the order in which changes in polling error occurred. That said, I don’t believe that explanation can account for how similar the geographic distribution of polling inaccuracy is to 2016.

So Florida and the Midwest are horribly wrong, but other locations aren’t as bad | Nate Cohn

Right. However, it may explain why national surveys are substantially more skewed on average, even if some of these underlying biases still persist at the state level.

You claimed that your 2019 Biden-Trump polls were close to the real outcomes. Isn’t there also a chance that the polls were incorrect, and that in 2019, Trump was genuinely ahead of Biden, and that coronavirus enabled Biden to come back and win?

Yes, that is entirely feasible. And I believe that if the final findings and polls are taken seriously, the Trump presidency will have to be rethought significantly. One of the problems with polls being incorrect, Isaac, is that you never know when they were correct or incorrect. To be clear, ambiguity occurs at all times. People assume that if they get the polls right in an election, they’ll be correct all the time until they’re next wrong. However, you never know if that is real. I’d say the same thing again this year. It’s likely that the polls represented President Obama’s popularity at various stages during the last four years. It’s also difficult to identify when they were correct or incorrect.

For example, everyone knows that the last Des Moines Register/Selzer survey, which showed Trump ahead in Iowa over the weekend, was one of the few accurate polls in this race. However, the previous two Selzer surveys in Iowa revealed a razor-thin contest, with [Democratic Senate candidate] Theresa Greenfield leading. So, what are we to make of that? Are we expected to think that the race swung dramatically in favour of the President at the end and that it was largely overlooked? Are we expected to assume that Selzer was just lucky, that if they had done five polls at the end, three of them would have shown Greenfield ahead just like the previous two, and she was just lucky on the final one? I’m not sure. It’s difficult to make sense of all the information you’ve received after learning that they were wrong in the end.

What did you hope to convey to readers through polling during an election about the race and politics? And have you had to reconsider that, given the amount of polling mistake we’ve seen in recent elections?

That’s a fascinating question, and I believe everyone uses political polls for different reasons. Some people conduct surveys primarily to anticipate election results, while others do them simply to gain a better understanding of the country’s sentiments. Because we live in such a polarised society, I believe that using polling to forecast election outcomes is extremely difficult at this time. In my lifetime, I’ve witnessed presidential election results ranging from Bush winning by two votes in 2004 to Clinton winning by eight votes in 1996. And because the range of conceivable outcomes or feasible outcomes is so narrow, and polls are never going to be perfectly correct, there’s virtually no election where polls can provide a final and decisive answer as to which party will win the presidency.

For example, the FiveThirtyEight model is primarily an exercise in measuring how inaccurate polls may be. And, in this period, there will never be a Presidential election in which the FiveThirtyEight model informs you on Race Day that the opposing side has no chance—at least not in a competitive national election. That has been the case for quite some time. What hasn’t been true in a long time is that the polls were also wrong about the election’s underlying storey.

For example, the majority of the pieces I write are about demographic changes. And I believe that those demographic changes are quite important because they have a significant impact on how our democracy operates. Political campaigns, activists, and politicians all strive to figure out what policies they want to support and what techniques they should use depending on their perceptions of the electoral benefits and drawbacks of various options.

Let me use my own favourite as an example. After the 2012 election, I believe Barack Obama and establishment Republicans such as Marco Rubio and Jeb Bush supported immigration reform because they believed Hispanic voters decided it in Barack Obama’s favour based on exit polls. As a result, getting those demographic trends right and accurately conveying the storey of these elections has a major impact on the course of our country’s politics. And the polls in this election did not adequately reflect it.

They claimed that Joe Biden was doing far better than Hillary Clinton among white voters. They were mistaken on that point. He did slightly better, but final surveys from some of the country’s most prestigious polling firms showed a virtual tie among white voters countrywide. It was not to be.

And if you can’t convey the tale of an election at the conclusion, the democratic process has severe flaws. Because politicians must represent the will of the electorate in a democracy, and if you can’t understand the will of the electorate at any given time, neither can our politicians. As a result, the public may be dissatisfied with what politicians attempt to accomplish on their behalf. As a result, I believe it is a big issue that the polls were so inaccurate this year.

The alternatives to polls, on the other hand, are not particularly good. As a result, we’re in a difficult situation where we have something that we believe is severely wrong, possibly so flawed that it could cause individuals to make decisions they shouldn’t make. On the other side, there are no viable options.

One counterpoint is that we live in a republic, and politicians do a lot of things to try to reflect what their constituents think—and they could do a lot less.

I believe there is a strong case to be made that if we could live in a world where elected officials ignored the will of the electorate for as long as they were in office, and we had a true republican form of government (where representatives did what they believed was right regardless of electoral consequences), I believe we would be a better country. I’m just not sure.

But the second point I’d like to make is that our government’s basis is built on democratic legitimacy, which is founded on the belief that people’s voices are heard in Washington and state capitals across the country. And part of the reason Donald Trump was elected President was that a large number of individuals did not share those sentiments.

My own fear is that removing polls from an election campaign would result in coverage that was no less horse-race-y, but instead relied on even less scientific criteria. They would be more mislead if we didn’t have polls and relied just on yard signs, social media participation, and crowd size, because Joe Biden is more likely to win by approximately five percentage points rather than losing badly. And all of these other indicators would indicate that Trump would win by a landslide.

There isn’t a good alternative to public polls, in my opinion. I don’t believe there are any other ways to gauge the opinions of such a diverse country. Without polls, I believe we would mostly regard the opinions of ourselves, our neighbours, and our like-minded friends. As a result, we require tools to communicate with people who are extremely different from ourselves in order to fully comprehend our country. On-the-ground reporting, in my opinion, is insufficient. Face-to-face polling is possible, although it isn’t all that effective.

However, I believe the more pressing issue is coverage of horse races. I used to work in the public policy field. I worked for the Stimpson Center, a think tank. And I used to have this naive assumption that data journalism would diminish horse-race coverage because reporters or others would realise that so much of the outcome of our elections is determined by underlying fundamentals such as economic development or whatever. And if that were the case, we wouldn’t have to pay attention to every blunder of the day since they wouldn’t mean all that much. Because, in my opinion, the only justification for worrying about the day’s blunder in the extent that we do is if it has an electoral ramification.

And, if we got to that conclusion, we’d talk about the things we thought actually mattered and were in the reader’s best interests, such as discussing policy. That is not how I believe it has turned out. I believe it has been an afterthought to traditional staged journalism, which, whatever its other merits and flaws, I do not believe was special or superior to polls in predicting election results. And, as far as I can tell, a lot of that journalism was based on reporting what internal polls said, which were just as terrible as public polls in this election.

Nate Cohn Should there be more polls, but fewer?

I’ve long believed that instead of everyone conducting their own surveys, media companies should collaborate on them. It’s what we do for the exit polls. And we don’t need six different national surveys a month from each of the big news companies, all of which say essentially the same thing, with the majority of the variance due to noise rather than methodological decisions. I also believe that doing so would allow for better analysis and the collection of bigger sub samples, among other things. To comprehend public opinion, I don’t believe we need a nationwide poll every day. I believe that the demand for information about how things have changed in our political moment is so great that it’s difficult to imagine a world in which polls are only released once a month or so.

The New York Times’ “needle,” which this year charted each candidate’s odds in Florida, North Carolina, and Georgia, provoked a lot of criticism, stress, and rage, as it always does. How did you think it went?

The Needle, in my opinion, did admirably. In 2016, I think it did a fantastic job. If you consider it solely from a journalistic sense, it scooped the world in 2016 on Trump being the favourite to win the election. In 2020, I believe Trump was the clear winner in Florida, the favourite in North Carolina although trailing by nearly double digits in the tabulated result, and Joe Biden was the clear favourite in Georgia, despite the fact that it occurred later in the night.

In many cases, I believe that people’s outrage over things like the needle or incorrect surveys is higher than their outrage over other types of journalistic errors. So I’m wondering whether we might reconsider since, while serving readers isn’t the only value of journalism, it is one of them.

Yeah. I’m completely open to that concept, if only because I believe the needle is the single best live forecast that has ever been published, and is responsible for breaking some of the most significant stories—at least in terms of electoral stories—in the last several cycles. So, if that’s the case, we’re probably doing something incorrectly. I’m not sure if it’s a matter of likelihood. I’m not sure if it’s P.T.S.D. on behalf of certain Clinton supporters from 2016. I’m not sure if it’s about the statistics or the visual display. But I believe there is something about how people perceive the needle that I don’t agree with.

Look, we put a lot of effort into it, and I believe we should be quite proud of it. It’s tough to put into words how pleased we are with how it’s turned out. And it’s quite annoying when other people don’t recognise it, and I’d like to alter that. And that could mean modifying how it’s represented, whether graphically or numerically, I’m not sure. But it’s something I’m willing to consider.

Joe Biden’s candidacy can be viewed in two ways, in my opinion. The first is that, even if he wins by a healthy-ish margin in the popular vote, the margin in important states was close, and hence, despite the pandemic, another Democrat might have lost this election. However, even with the pandemic, Trump was able to secure all of these Republican voters, and the country is, as you point out, quite set in certain ways. What are your thoughts?

From my perspective, the Biden campaign was built on the assumption that the public did not want to re-elect Donald Trump as President, so you should give them the least offensive candidate possible and you will win the election, and that you can only risk losing the election by choosing someone more controversial. The first half of it, I believe, has been complicated by this election. There was no clear majority of the electorate who did not want Trump to be re-elected as President.

The second element of that formulation, that choosing the least contentious candidate maximised your chances of winning, has not necessarily been invalidated by this election. We’ll never know if Elizabeth Warren or Bernie Sanders would have cost Democrats six-tenths of a point in Wisconsin, in the traditionally Republican suburbs of Milwaukee, where Joe Biden will win this election. We’ll never know for sure. However, I do not believe that the election results prove or contradict that possibility.

In terms of the popular vote, this election was much closer than I anticipated, and Donald Trump was clearly on the verge of winning. That said, I believe it’s difficult for journalists, who are supposed to be engaged with the idea of talking about what “the people” want, to forget that Joe Biden winning by a larger margin than, say, Obama defeating Mitt Romney, is a significant achievement in our partisan environment. How do you strike a balance?

That’s a wonderful question, and I believe the actor plays a critical role here. If you’re a politician aiming to win a congressional district, you’d be quite right in focusing on the opinions of your constituents. If you’re running for President, you’ll concentrate on the states that decide the Electoral College. If you’re a political movement, such as progressives or alt-right conservatives, or whatever, and your objective is to gain political power in this country, you must concentrate on doing it within the framework of the current system. If you’re talking about the public will and what Americans believe in broad strokes, you’re talking about something quite different.

When we cover progressives or whatever, and we’re debating whether Democrats should support Medicare for All, it seems to me that much political journalism is focused on the quest of power. It isn’t a question of what the American people believe. Is this anything that will assist Democrats in seizing power and enacting their favoured agenda? In which scenario, focusing on electoral outcomes or voter sentiments in the most relevant districts and states is justified. However, I believe you are correct that the national vote conveys a different storey if we are simply trying to characterise the American people in general.

Also Read:

Next articleRussia Ukraine: border, raising concerns, Russia told the US