Election polls are actually getting better

Insider

When pollsters failed to see the surge of support for Donald Trump in 2016 and 2020, it seemed like a low point for the industry. But in fact, surveys are getting more accurate, as a look through the archives shows.

Transcript

00:00That says it all. It's the Trump.

00:02Trust in polls and pollsters hit a new low in 2016,

00:07when most showed that Hillary Clinton was more likely to win.

00:11How did the polls lead us astray?

00:13It just came out of nowhere.

00:15And then again in 2020.

00:17The polling was wrong. It was wrong again.

00:20Fast forward to 2024, and few are daring to make a call.

00:25I mean, this is the closest race in American history.

00:27It is really neck and neck.

00:29In fact, there is only one perfect poll,

00:33and that's called an election.

00:35Believe it or not, the polls are not becoming

00:37less accurate over time.

00:38So how did a once-respected industry

00:41that grew to a value of $20 billion come to this?

00:45And can it win that trust back?

00:47Fame statistician Dr.

00:49George Gallup tells Washington from week to week

00:51what the nation is thinking.

00:54It's been more than 80 years

00:55since a man named George Gallup conducted his first poll.

00:59He used a simple but effective technique

01:01to survey the opinions of the American public.

01:045,000 men and women mathematically chosen

01:06from every walk of American life

01:08to make up an accurate microcosm of the entire U.S. population.

01:12Before this time, surveys had been a lot less scientific.

01:17How are you going to vote?

01:18I'm going to vote for Roosevelt

01:19because I think he's going to bring back the ban.

01:22The most reputable forecasting of the early 20th century

01:25was conducted by the Literary Digest.

01:28It asked readers to mail in sample ballots.

01:31In 1936, though, the Literary Digest backed

01:34Alf Landon to win the presidential election.

01:37But Roosevelt ended up winning by a landslide.

01:41The great American public has its say at the polls,

01:44and the result is a Roosevelt victory of amazing proportion.

01:48George Gallup had predicted that result

01:51by surveying a wider cross-section of Americans.

01:54Today, the true cross-section survey

01:56has become a trusted political barometer.

01:59The Literary Digest went bankrupt,

02:01and Gallup's polls became a hit.

02:03The Gallup surveys of the 1930s

02:06ushered in a new era for polling and for voter confidence in it.

02:10Back then, Americans were not only willing

02:12but excited to talk to pollsters.

02:15It used to be such an honor to be included

02:19as one of the special households

02:21who got to speak for thousands of Americans.

02:24In 1948, Time magazine even described Gallup

02:27as the Babe Ruth of the polling profession.

02:31That same year, Gallup backed the wrong candidate

02:33in the presidential race, but his reputation survived.

02:37We know that no better method has yet been found

02:40for measuring political trend.

02:43Those were the days when polling was revered,

02:48and technology had to catch up with polling,

02:50which it finally did in the 1970s.

02:53Once telephones were widespread in people's homes,

02:55pollsters didn't have to go door-to-door anymore.

02:58But there was a downside.

03:00Telephone response rates began to fall in the 90s

03:02with the invention of caller ID.

03:05Some companies eventually transitioned to online surveys

03:08that voters can respond to whenever they want.

03:11And by the early 2000s,

03:13there were more than 5,000 polling organizations in the U.S.

03:17That's when a baseball statistician named Nate Silver

03:20analyzed a number of polls and fed them into models

03:23that simulated the elections.

03:25He used them to come up with forecasts

03:27and give the public a glimpse

03:28of what Election Day in 2008 could look like.

03:31And may God bless the United States of America.

03:34Silver was able to correctly predict the winner

03:37in every state except for one.

03:40Basically, he made polls cool again.

03:43But in the 2016 elections, all of this changed.

03:47Look at how close we are to the polls.

03:49Look at how close he is.

03:51Right now, he has 257 electoral votes.

03:54He needs 13 more.

03:56It wasn't just a tough night for the news industry

03:58but for pollsters.

04:00I had chills when it became clear,

04:04whatever it was, 9, 10 o'clock Eastern time,

04:06that this was no layup for Hillary Clinton.

04:09It was a huge shock.

04:11Dr. Wong, you tweeted recently

04:13that you were so sure of the result

04:15that you'd put it up on the screen, eat a bug.

04:18A lot of people were wrong,

04:19but nobody else made the promise I did.

04:21The polls basically got trashed.

04:24But that wasn't totally fair.

04:26Polls are not meant to be a exactly perfect prediction

04:29of what the eventual vote breakdown is going to be.

04:32And it's normal for the polls to be three to four points off.

04:34The polls did miss, but not by an unusual amount.

04:38They were off by about 4.8 percentage points.

04:42That's within half a point of the average error

04:44for election polls over the last 50 years.

04:47On election day, pollsters said that Clinton was leading

04:50by about 3.2 percentage points.

04:53Clinton did end up winning the popular vote,

04:55but by a smaller margin of 2.1 points.

04:59And as we all know, she lost the election.

05:01What happened was that Trump was able

05:03to win the electoral votes in a lot of states

05:05where Clinton was leading by a small margin.

05:08But fine margins can make a huge difference

05:10in the electoral college.

05:12Part of the problem was that pollsters were too focused

05:14on talking to those who seemed like likely voters

05:17based on their past voting history.

05:19The people who ended up voting for him

05:21are perhaps low propensity, low turnout voters

05:24who would not pass a likely voter screen

05:26or just not be reached by a poll at all,

05:28but did ultimately turn out to vote for him.

05:30They also didn't factor in education levels.

05:33College-educated voters were more likely to vote for Clinton

05:36and they were also more likely to respond to surveys.

05:40So by not adjusting the sample, pollsters ended up

05:42with a forecast that overestimated support for Clinton.

05:46You can end up in a situation

05:47where you just miss a big part of the electorate

05:49and end up underestimating a campaign support.

05:52Ultimately, the results didn't match the scenario

05:54the media had hyped up the most.

05:57But the reputational damage seemed to be done

06:00and the public started to lose faith.

06:02Get it done! Get it done!

06:05Pollsters tried to implement a couple of changes

06:07in the 2018 midterm elections,

06:10like factoring in educated voters,

06:12and the forecasts turned out to be correct.

06:15But when 2020 came around,

06:17lack of trust was still one of Jay's biggest challenges.

06:20Voters were reluctant to answer his questions.

06:23How do we draw a sample that actually is random?

06:28And we as a profession are struggling right now

06:32with how to do that

06:33because people do not want to be bothered.

06:36They have ways of communicating that are invisible to us.

06:40Trump voters are less willing to participate in the polls,

06:43but not because they're shy.

06:45If you have been conditioned for four years

06:49by tweet after tweet that says,

06:51that poll that shows me, Donald Trump,

06:55trailing is a fake poll,

06:57you may be reluctant to talk to a pollster.

07:01And any poll that the fake news gives you,

07:03because the real numbers are higher,

07:05but any poll they give you is a fake.

07:07Pollsters did correctly predict

07:09that Joe Biden would win the 2020 election,

07:12but they again underestimated support for Trump.

07:15One of the states that first started reporting its votes

07:17was Florida, where polls had Biden up by a few points,

07:20but it was still expected to be a very close race.

07:22As results started coming in,

07:24we saw that Trump was overperforming the polls again.

07:27News organizations chose to represent

07:29their predictions in new ways.

07:31The New York Times published a table

07:33that included a column that read,

07:35if polls are as wrong as they were in 2016.

07:39And even FiveThirtyEight showed a number of maps

07:42to tease the idea of multiple scenarios.

07:45Overall, the polls did all right.

07:47Between response rates,

07:48coming up with a representative sample,

07:50and accounting for the pandemic,

07:52given that, it is pretty impressive

07:54that polls did predict the right winner.

07:56They were off by four points,

07:57and that's normal by historical standards.

08:01More than 100 years after the first poll was conducted,

08:04the industry has had to reinvent itself all over again.

08:09Would you mind telling us

08:09who you're going to vote for and why?

08:11I'm going to vote for Governor Roosevelt

08:13because I think he'll bring back prosperity.

08:16I think Hoover deserves another chance.

08:18I'm sure he's the best man for that office.

Category

Transcript

Recommended