While polls often overestimated support for Republicans as well, the average survey overestimated Democrats significantly. For any given Senate poll released in August of 2024, 2022, 2020, or 2018, you had a much better-than-even shot at predicting Republicans would beat it. About 70% of August polls in the past 8 years have overestimated support for Democrats.

Ruffini’s chart is in many ways unsurprising. The window contains both the 2024 and 2020 elections, years we know surveys had too many Democrats in them, and where preferences moved right in the final months. But it also contains 2022 and 2018, when polls did much better. In 2022, polls on Election Day actually overstated support for Republicans, bucking the previous cycles’ trend.

But the big question this piece prompts us to ask — indeed, prompted dozens of you to email me about — is this: so are the polls overestimating Democrats again this year?

The truth is, Ruffini’s argument is right in some ways and wrong in others. He is right to point out that polls early are often “wrong” (only if you view early polls as forecasts of future preferences instead of snapshots of opinion at the time they were taken), and the election forecasting model I built for FiftyPlusOne.news goes to great lengths to treat polls with the proper amount of uncertainty/skepticism.

But the implication or the piece — that polls are likely biased toward Democrats right now — is not warranted by looking at the historical data. Ruffini only has 4 cycles of polls in his analysis. But I have polls going back to 1998, so in this week’s Chart of the Week, I show that bias in polls tends to change randomly from year to year. Like taking the under on low-probability bets, predicting polling error from previous cycles’ misses works until it doesn’t.

This is post is free to read, but if you want to support my independent data-driven political journalism and get premium posts like these sent to your inbox 2x a week, become a paying subscriber to Strength In Numbers today. Members also get early access to data products, a private community on Discord for our political data nerds, and an ad-free newsletter experience.

Upgrade to support Strength In Numbers

If a paid subscription is out of reach, the next best thing you can do is to share this article with a friend. Organic recommendation is the #1 driver of new readers here at Strength In Numbers.

Share

“Unskewing” the polls usually increases error

I have taken FiftyPlusOne’s dataset of all horse-race general election polls since 1998 and filtered it down to polls released by August 31 in each year. Then, I calculated rolling 45-day polling averages for each competitive Senate race in those cycles and compared those polling averages to the November results in each race.

The table below shows the average bias of polls in the average race by year, going back to 1998. It also contains, for each year, (a) the average bias of polls on Election Day and (b) the average bias in polls released in August in the prior 4 election cycles.

As Ruffini found, those August polls have been overestimating Democrats recently. But note (a) that this wasn’t/is not always the case, and (b) polls often improve in predictive performance as we get closer to November.

What’s really striking from this table, though, is how poorly lagged cycle bias predicts future poll misfires.

The lagged average bias of polls in year Y-8:Y got the direction of bias in year Y right in ten of fourteen cycles. But it missed the magnitude by an average of 2.5 points, and the direction about 30% of the time. And the predictive failures one would make using the heuristic “polls will be as biased as they have been recently” are not small ones. In 2014, the four preceding cycles pointed to polls understating Democrats by two points; August polls that year overstated them by seven. You would have created a 9-point error in the wrong direction in your poll or election forecast if you had applied the lagged error to the average survey.

Similarly, in 2016, the 4-cycle lagged average bias suggested the polls were essentially clean, and they turned out to be off by nearly six. Back in 2002. You would have predicted a pro-Democratic polling environment on the historical data, but surveys had a two-and-a-half-point Republican tilt instead.

Plotting the same data above makes the problem inherent in predicting poll bias easy to see. While two variables — observed poll bias and lagged bias — are generally related, they are not highly correlated, and there are some very big differences.

Each election year occupies the point in the graph above corresponding to its cycle’s lagged average bias on the horizontal axis and its observed August bias on the vertical. If past bias predicted future bias, the years would line up along the dashed diagonal. Instead, they scatter.

The fitted line slopes upward at 0.57 — meaning a one-point pro-Democratic tilt in the last four cycles came with about half a point in the next one. But the standard error of that slope is 0.38, which means we cannot statistically distinguish it from zero. Zero is what you would get if the past told you nothing about future polling bias at all.

But the wider shaded band is the real kicker here. That is the 95% prediction interval — the uncertainty interval around predictions of future bias given lagged bias over the past 4 cycles. It is huge. Knowing what polls did in the four previous cycles narrows your expectation for the next one by essentially nothing.

Ruffini’s post is about August polls, but for the record, the serial correlation of poll bias is even worse on Election Day. Bias by Nov tends to even revert somewhat from the previous cycle.

Check out 2022 there, on the far bottom right of the graph. The lagged bias then predicted a 3-point overestimation of Democrats in Senate races, but the bias ended up by a 3-point miss in the opposite direction. If you had adjusted polls for past bias in 2022, you would have massively messed up your election predictions based on those polls.

And this is the finding that really matters — because if you’re going to put your thumb on the scale for a party, you want there to be a close to zero chance of causing a larger error than if you had left the polls alone. We can test this too.

“Unskewing” also introduces high “risk of ruin”

I went back to my historical dataset of polling averages, took the lagged bias values from the first table of this post, added those lagged direction biases to the polls in each year, and then calculated the average bias of the resulting polls. Then I did the same thing, assuming no average bias in the polls.

The chart below shows the distribution of signed error in Senate polling averages under each scenario, and for polls as of August and November.

Note that the adjusted polls do not systematically outperform the unadjusted polls — and in many cases actually do worse. Since polls tend to do well or poorly across the country by similar amounts each cycle, this thumb-on-the-scale adjustment of polls also creates the problem that a forecaster releasing adjusted numbers tends to increase all their errors across all races at the same time.

The 2022 election is a good example of this. If one were to have calculated averages in each Senate race and then subtracted out the bias in polls from 2014-2020 from them, one of have pushed all the polls to the right by exactly 3 percentage points. This would have increased your error in 8 out of 8 key Senate races, and called 4 races for Republicans that polls said Democrats lead in.

If that sounds familiar to you, it’s exactly what RealClearPolitics did in 2022. I wrote about how stupid this was while it was happening here. Back then, I wrote:

These results suggest that if RCP repeated its adjustment procedure for all Senate elections since 2002, they would have ended up with more biased polling averages than if they had stuck with the raw average in 7 of the last 10 election cycles. The only years in which their adjustment would have made their averages more accurate are 2010, 2012 (barely) and 2020

For Senate races in August, adjustment would have improved your forecasts in 9 out of 14 of the last elections and worsened them in 5/14. By Election Day, the results flip, and an adjustment would have cost you in 10/14! In those races, it would have been better for you to leave the polls alone and focus on measuring uncertainty, not predicting bias.

Share

Why 2026 may be different

Zooming out before conclusion, it’s worth a couple of paragraphs to think about the mechanisms of “error” here. Looking back at Ruffini’s chart, why was there such a reliable shift from August to Election Day in 2018-2024?

Well, one thing to say is that early polls are not meant to perfectly predict election outcomes. Voters’ preferences can change over the campaign as they tune in to information and socialize about key races. Some of this “error” in August polls is likely measurement error, I grant that — but it’s also the case that it’s truly just pretty early in the campaign still. Most people are checked out until after Labor Day.

But another plausible reason is that early polls tend to sample adults or registered voters instead of likely voters. But as Election Day approaches, pollsters tend to make the switch to the more accurate voter screen.

In the past few cycles, Democrats were doing better in RV and A polls than in polls of likely voters. So the LV switch produced a natural curve to the right that made it look like Democrats were losing ground — when in reality, their support in early polls was inflating because they sampled the wrong populations.

But, as Ruffini’s own polls have shown, this is not the case for 2026. Democrats are doing about a point better in LV polls vs adult and RV samples this year. So one could even argue we should naturally see the opposite drift this year as pollsters switch to LV screening.

In 2022, you also had Dobbs in the summer, which delayed but did not eventually disrupt the typical midterm dynamic of the party in power losing ground as Election Day approached. There is no Dobbs in 2026 disrupting that cycle, so the data points from that summer may be useless in anticipating potential poll bias.

Still, none of this should be read as me arguing that the polls are all fine and affirmatively won’t be biased come November. Early polls will not be a perfect prediction of the November election results — that much we know pretty much for sure.

This is a good reason not to take the polls at face value and to instead use something like the FiftyPlusOne.news election forecast that (a) simulates uncertainty in polls according to this exact historical distribution of error and (b) combines the polls with other predictors of election outcomes, including “the fundamentals” (past district history, incumbency status in a seat, etc) and race ratings.

In sum, it is undisputed that polls have been trending in a pro-Democratic direction recently. But that does not mean that they are reliably biased in a knowable direction, such that you can subtract the error in advance and arrive at an unskewed result. Ruffini argues that Senate polls are “always worse in the summer.” This is true with his limited post-2018 data, but it is not supported by a full historical study back to 1998.

Adjusting your polls in the way Ruffini’s findings imply you should would have made your forecasts worse in many key races — including in 2022, when early polls forecast Democrats would hold onto their majority in the U.S. Senate, which they ultimately did.

Given the stakes of elections today (and frankly, the reward in making overconfident forecasts months out), it is tempting to try to forecast future polling misfires using those in the recent past. But the historical evidence suggests this is a fool’s errand. After all, predictions are hard, especially about the future.

If you believe in the importance of data-driven coverage of polling and public opinion, the best way to support my work is with a paid subscription to this newsletter.

Upgrade to support Strength In Numbers

Related Articles

Subscribe now

Here are the big takeaways:

• A firm called “Median Strategies” faked polls in California and Wisconsin. The firm told the LA Times it had invented its results as a “social experiment,” then scrubbed its anonymously registered website once reporters started asking for methodological details. Mary explains why FiftyPlusOne never reported on this organization’s “polls” and the screening questions she asks every pollster before she shares their numbers at 50+1.

• A pollster called The Public Sentiment Institute admitted to reassigning respondents’ vote choices to boost numbers for one candidate — who coincidentally paid for the company’s polls, which it didn’t disclose. TPSI says they “leaned” respondents who had selected voting for Jay Collins toward James Fishback — whose campaign, it turns out, had paid them about $1,500 shortly before the polls came out. Collins finished in the mid-20s, and Fishback got 10%.

• Fake polls have always been around. But it’s easier than ever to conduct surveys that are both real and unprofessional or low-quality. A firm called Research 2000 faked Daily Kos’s daily Obama approval tracker for over a year before they got caught. Now, in 2026, AI tools make it trivial for enterprising young coders to fabricate release paperwork and even topline and crosstab data — and anonymous prediction markets create an avenue for payoff. Mary says that social media accounts that collect and republish polls should only publish verified results, and link to their sources (instead of posting screenshots of polls to go viral). Better yet, just go to FiftyPlusOne.news for all your polling needs.

Thanks to everyone who joined us for the weekly live stream on Substack, and we’ll be back next week!

As a reminder, paid subscribers to Strength In Numbers get to participate in our live Q&A! If you missed our video livestream, you can watch it by clicking play on the web version of this post at gelliottmorris.com.

You can also subscribe to us on your favorite podcast app to listen on your own time. And if you do listen via one of those apps, please drop us a five-star rating and review if you feel we’ve earned it — it really helps people discover the show!

If you value this work and want to help keep it going, please consider becoming a premium subscriber to Strength In Numbers. Paid subscriptions support this podcast, the newsletter, and the time it takes to do original data-driven analysis of politics and elections. Paying subscribers also gain the ability to send in questions during our live streams, so you can directly shape the conversations we’re having on the podcast.

Upgrade to support Strength In Numbers

You can read the transcript of our conversation by clicking the headline of this article to take you to the web version of the podcast, then clicking the button just below the byline that looks either like a piece of paper or is labeled “Transcript,” like so:

If you’re a reader of Strength in Numbers and haven’t yet subscribed to David’s newsletter, head to the-downballot.com.

Subscribe to The Downballot

And if you’re coming from David’s site, subscribe here to get the numbers behind the news.

Subscribe to Strength In Numbers

This post is part of The Data Lab, a new section of Strength In Numbers for extra charts, general polling wonkery, and methods work that doesn’t fit into the normal editorial vertical. If you’re the type of person who wants to know how our polling is done or reads election forecasting methodologies, the Data Lab is for you.

Emails for posts on the Data Lab vertical are off by default, but you can easily toggle them on here. If you have a post to contribute to the Data Lab, email me.

Most Data Lab posts are free, but the occasional deeper ones are for paying subscribers, whose support makes all of this possible.

Upgrade to support Strength In Numbers

The folks behind the Marquette University Law School poll have responded to my analysis of last week’s polling miss in the Democratic primary for governor of Wisconsin. As a reminder, the final Marquette poll of the race from July 22-27 had Francesca Hong winning 38% of the vote and David Crowley, who ultimately won with 40% of the vote, at 7%. This is a 33-point miss — a “remarkable difference between poll and outcome,” per the director of the poll, Charles Franklin. In margin terms, this is a 31-point miss.

As a second reminder, my analysis looked at the microdata from a non-profit I did some poll methods work for previously called State Navigate, whose August poll had Hong at 44% and Crowley at 20% — a 20-point miss, about 50% less than Marquette.

The explanation from Marquette of their miss is here. Part of their explanation is also that their poll was taken early, before a lot of late consolidation to Crowley. That’s plausible, but also, 31 points is a lot of consolidation. This is one of the hard things about conducting primary polls in general, though; preferences are very fluid and tend to move a lot very late, especially when candidates are entering and exiting the race.

I don’t want to get into a line-by-line back and forth, but I do want to put their opening here. Marquette’s piece begins as such:

G. Elliot Morris argues that the polls in Wisconsin underestimated David Crowley’s support in the 2026 Democratic primary because they included too many young voters (who mostly supported Francesca Hong) and too few older voters (who mostly supported Crowley).

I think a more accurate reading of my article is that the State Navigate survey was primarily biased because they included too many young voters (with a target for the under-30 population ranging from 9% in their August poll to 17% in July. Additionally, in my article I gave three reasons why the polls were off, not just one:

• Most of the miss in polls in Wisconsin is attributable to faulty demographic targets (too many young people). This inflated Hong’s vote margin by somewhere between 5 and 10 points.

• My best guess is that the race moved 6-10 points toward Crowley after pollsters released their final surveys.

• Non-ignorable non-response within demographic categories likely further inflated Hong’s vote margin by 2-5 points.

Fair enough, I did say “in polls” there, though the article was mostly about State Navigate. I think this probably made Marquette think I was attacking them and hurt their feelings, though, which was not my intention. But note I did attribute the lion’s share of the “error” to late movement, as does Marquette’s John Johnson:

I would not describe late movement as leading to “error” in our survey, since we were measuring the electorate as of July 22-27 and not predicting what it would be on August 11.

The remainder of Johnson’s article is a good deep dive into why it’s very hard to know what the age composition of the likely total electorate in Wisconsin is. That’s because the state of Wisconsin does not make age of registrant available for commercial voter file vendors, so you get the “vampire” problem State Navigate termed and which I described in the footnotes of my original article, and in this follow-up.

I’m pretty sure Johnson didn’t read the footnotes or follow-up because I did a sensitivity analysis of the age makeup of the Wisconsin electorate that answers his objections outright. I explained that we didn’t know from the WI file what the unknown % exactly was, that polling data provided a benchmark of about 50% - 75% of the entrants with unknown ages being in the 18-30 camp, and then tested what the analysis would show assuming those percentages. I demonstrated that the poll bias was not linearly related to the age composition; that is, even at a high 18-29 voter share, the weighted State Navigate polling data was still a lot better than what they published in August. I’m sure it would make a big difference vs what they published in July,

I explained, for example, that even if you weight the State Navigate data to a 9% target for ages 18-30, you still get a big Hong overestimation (this data is from August 7-8, the week before the election):

Importantly, I also showed that weighting to a rough guess of the ideological composition of Wisconsin Democratic voters, via 2016 primary exit polls, gave a much more accurate picture of the race (that’s the solid line above).

But none of this answers the big question: What share of the Wisconsin likely primary voter universe is made up of under-30s? We will never know for sure, so we are left guessing — and, importantly, weighting polls to those guesses, which is why sensitivity tests of the kind above are so useful and why pollsters should publish them.

For some informed guesswork, Johnson produces this table of estimates of all registered voters and general election voters using data from the Current Population Survey. It shows that 14% of self-described 2024 general election voters in Wisconsin were ages 18-29.

This appears to be slightly less than the benchmark Marquette weighted to (18% in their published tabs). I think this is because their poll weighted to registered voter benchmarks, and then filtered to the population of likely voters based on their stated intent. (This introduces a lot of noise because people are really bad at estimating how likely they are to vote.) I have asked Johnson on Substack to give me the exact weighting targets and will update this paragraph when he responds.

Unfortunately, I’m pretty sure both of these estimates from Johnson are too high. For one thing, a poll weighted to a lower share is more accurate in predicting the winner, even if you adjust for ideology and for late deciders favoring Crowley. (But maybe that’s just because those weights deflate Hong supporters, but still give an unrepresentative look at the electorate. Fair enough.)

The problem with Johnson’s estimates is that he is looking at a different population than I was. For re-weighting the State Navigate data, I was targeting the population of Wisconsin voters who have cast at least one ballot in a Democratic primary for any office since 2016. Johnson is, in contrast, producing an estimate of the statewide population of likely November presidential general election voters. The latter population is going to have a lot more young people in it, because primary voters skew old.

To demonstrate that point, I have loaded up the L2 voter file for Minnesota, which does not have missing data for voters by age, and is demographically one of the most similar states to Wisconsin. According to L2, the statewide share of registered voters in Minnesota who are between the ages of 18 and 30 is 18%. However, the share of likely Democratic primary voters who are between 18 and 30 is only 11%. That’s a 7-point difference. Additionally, about 14% of validated 2024 voters were 18-30 — 3 points higher than the likely primary electorate.

And according to the Census, Minnesota has about 2 points more voters who are 18-30 than Wisconsin does. (Wisconsin voters are older than Minnesota voters.)

If we take Johnson’s estimate that 14% of Wisconsin 2024 voters were ages 18-30 and subtract 3 + 2 from it, we get 9%.

This is the same target I used in the updated analysis I posted last Thursday, which “controls for the vampires.” In that analysis, I produced the following breakdown of error in the State Navigate data, using the 9% 18-30 target:

1. 10 points having too many young people people in the poll / other weirdness in the State Navigate weights

2. At least 5 points of late movement (5 is what we can detect with the recontact) and plausible up to 10

3. About 5 points of non-response bias

Adjusting for the misspecified age composition gets you within the 13 percentage point average error for the margin between two candidates in primary polls. That would indicate the primary polls and Wisconsin were not any worse than average, after adjusting for what looks like maybe a simple mistake or extra extrapolation from Michigan on the part of State Navigate.

But you can eliminate most of the error in 1 and 3 by weighting your primary poll to a reasonable benchmark for ideological composition. We don’t know what the precise ideological composition of any group of voters is, so see Recommendation 3 in my original analysis. Time for some sensitivity analysis, baby!!

For a few more examples, Strength In Numbers subscriber and the former Secretary of State of the state of Oregon, Phil, calculated the following turnout and young voter percentages in recent primaries in the following states:

• NC - 21% overall turnout, 18-29s 7% share

• OR - 42% overall turnout, 18-29s 6% share

• PA - 22% overall turnout, 19-29s 8% share

• OH - 23% overall turnout, 18-29s 7.5% share

He wrote me:

With 35% overall turnout in WI, my educated guess is that the 18-29 year old share of the D electorate might be closer to 10%, plus or minus 2%.

In summary

1. I’m sorry if I hurt Marquette’s feelings.

2. I already conducted a sensitivity analysis on the youth composition of Wisconsin voters, which I should have just published up top in the first article so I didn’t confuse people, and the conclusions about error do not change.

3. Primary polling is very hard, especially in states where we don’t have good voter file data on age composition. People should adjust their expectations for accuracy downward, especially in polls taken weeks before the actual election (as in the case of Marquette).

4. Marquette’s alternate estimate of the age composition of Wisconsin seems too high; this is because they are not adjusting for the fact that likely primary voters tend to be older.

If I were you, I would just put this on repeat:

Primary polling is very hard, especially in states where we don’t have good voter file data on age composition. People should adjust their expectations for accuracy downward, especially in polls taken weeks before the actual election.

Writing for the New York Times last weekend, Nate Cohn argues “It can be hard to predict exactly which progressive candidates are liable to underperform, but Dr. El-Sayed has a lot of the hallmarks of the kind of progressive who has underperformed in the past.” I discuss this piece at length below the fold.

Because of columns from Cohn and others, most Democratic elites have adopted the belief that progressives always underperform moderates. Readers would not be blamed for believing that as well.

But I’m not here today to argue against the idea that moderates do better than progressives (despite the real evidence otherwise). Instead, I’m arguing for uncertainty — for even just a dose of epistemic humility from DC practitioners and media pundits about why voters are voting the way they are. If you take a look at the research on this subject, it becomes clear that we all know much less than the conventional wisdom asserts. And more than that, we know a lot less than pundits say about how voters will behave in the future.

So this piece is partly about El-Sayed in Michigan, and partly about how confident any of us should be in the new conventional wisdom about “electability” and moderation. The evidence on whether ideology costs a candidate votes is thin, contested, and importantly constantly shifting. As Cohn notes accurately in his piece, there is even debate about what counts as “progressive” anymore. As he shows in his examples, it has increasingly little to do with policy preferences.

Putting our Bayesian hats on, the honest accounting of all the evidence is that we can’t be sure of a penalty for ideological “extremism” in any one isolated race — and almost nobody in this argument is accepting that uncertainty.

But before we get into all that, it’s worth taking stock of the current and past polling in the Michigan Senate race. What do they say now about El-Sayed? What did the polls say about Stevens, an unabashed moderate — would she have fared any better?

Then, I dive into Cohn’s big example for how to be a progressive and not appear as one: Tammy Baldwin. The problem here, unfortunately for Cohn, is that Baldwin is seen as the second most liberal Democratic Senator. Ouch.

The rest of this Deep Dive is for paying subscribers. I write at least one post a week reserved for the members of this community who keep Strength In Numbers running. Upgrade to continue reading and support the time and data collection it takes to do analyses like this one.

Upgrade to support Strength In Numbers

What the polls actually say about El-Sayed and Stevens

Read more

This post is part of The Data Lab, a new section of Strength In Numbers for extra charts, general polling wonkery, and methods work that doesn’t fit into the normal editorial vertical. If you’re the type of person who wants to know how our polling is done or reads election forecasting methodologies, the Data Lab is for you.

Emails for posts on the Data Lab vertical are off by default, but you can easily toggle them on here. If you have a post to contribute to the Data Lab, email me.

Most Data Lab posts are free, but the occasional deeper ones are for paying subscribers, whose support makes all of this possible.

Upgrade to support Strength In Numbers

The feedback to my Thursday piece re-analyzing Wisconsin polling microdata has been overwhelming and positive. I have received more emails and texts about this survey than anything I’ve written in a while. I consider this a huge win, not only because I am a millennial/Gen-Z and crave your validation, but because it shows the utility of explaining the ins and outs of polling in public.

I did want to issue a short follow-up post about this. What about the vampires? This post is going in the Data Lab, because it’s for the real sickos out there.

Do not invite the vampires into your poll

The Wisconsin voter file is notorious for having a lot of missing and incorrect data. One of the variables that is particularly errant is the age variable. In their methodology report, for some added context, State Navigate wrote:

“many voters in Wisconsin have a birth year of 1800, and we don’t believe in vampires. These generally are newer voters who skew very young; about half of voters who we polled with a birth year of 1800 say they are in the 18-29 age bracket. Thus, if you were to simply take the voter file’s birth year data, you’ll get a Democratic primary electorate that is majority 65+ and has less than 2% of voters under the age of 30. Absentee requests for the Wisconsin primary (which includes requests for Republican ballots) are 5-6% 18-34. Given that this age group has been notorious for voting late/on Election Day, we find it hard to imagine a Wisconsin Democratic primary electorate that is <8% 18-29. Maybe it is; we don’t know for sure due to the ridiculousness that is the Wisconsin voter file”

Well, I am more than a little wary of using polling data that had too many responses from young people to make any sort of predictions about what age numbers would be on the voter file, so it is another opportunity for sensitivity analysis.

In the plot below, I re-raked the August survey with the under-30 target set by increasingly aggressive assumptions about how old the “vampires” are — from my baseline a priori estimate (unknown ages allocated in proportion to the known ages) up through State Navigate’s estimate (half of unknowns under 30) and past it (70%) — under both the L2C weights and the ideology-free L2B weights.

Notably, stepping up the share of voters who were <30 increases error linearly. Under the L2C weights (which account for ideology and income), the margin goes from Hong +2.4 at my baseline, to Hong +4.9 using State Navigate’s stated floor of an 8%-under-30 electorate, to Hong +6.6 if you assume half of unknowns are under-30, and Hong +8.4 at 70%.

Under the L2B weights (which don’t have ideology in the targets), the same exercise runs from Hong +13.8 up to Hong +19.7, nearly all the way back to the unweighted poll’s +23.8. The reason is simple: the young people who actually answered this survey were overwhelmingly Hong voters (likely too pro-Hong, as the non-response analysis suggests), so every additional young voter you add to the electorate model pushes the poll further from the result.

The adjusted weighted estimates I present — even at a 7-10% under-30 percentage — are still much closer to the result than the original weighted or unweighted numbers. Whatever the true age of Wisconsin’s vampires, the topline takeaways from my original analysis stand: The polls had too many progressives in them, which you can mostly but not entirely solve with weighting — especially if you don’t know the breakdown of the electorate by age! — and a large amount of late movement toward Crowley.

Share

Somebody get the garlic

But about the non-response problem: With the adjusted numbers and new age breaks, the weighted estimates do land farther away from the targets. Does that mean we should conclude there was more non-response among moderates after all?

Well, not entirely. Again, the big problem here is that the Wisconsin voter file can’t produce a good benchmark for age (or other) variables, so pollsters are left flying blind as to what to weight their data to. (The right move here, in my opinion, is to simulate what your poll would show under a bunch of different weights to varying targets, but I will save this for another post.)

To set the stage, here is what the results of the State Navigate poll in August would have been if you allocated undecideds and some minor-candidate voters to Hong and Crowley based on their flows in the July-August recontact study State Navigate conducted. This uses my adjusted weighting model and the maybe-realistic-but-we-don’t-really-know assumption that 50% of people on the Wisconsin voter file with a birth date in the year 1800 are actually under 30 years old (what the polls that had too many young progressives in them indicated).

What is really interesting here is that the base scenario is almost perfect on Hong’s vote share, but underestimates Crowley. That’s likely because there was an additional large swing in his direction after the final August poll wrapped fielding, which polls simply cannot capture. The observed flow to Crowley among supporters of the minor candidates in the recontact study was 1.5:1. I also calculated what the results of the poll would have been if you allocate those voters 2:1 to Crowley vs Hong:

Now, here is the same chart, but if you weight the sample additionally to a benchmark for ideological makeup of the Wisconsin primary electorate. (Remember, the State Navigate poll had a quarter of respondents identifying as Democratic Socialists, which we can be reasonably sure is too high.)

Weighting by ideology recovers about 8 points of vote margin for Crowley.

This means moderates aren’t responding as much to our surveys, right? Well, yes and no. We don’t know if “progressive” is the psychological quality that’s acting on voters to not make them vote. It’s likely something related to one’s ideology: like political engagement. I don’t have the year-vintage breakdown of vote history participation for Wisconsin voters in, say, 2018 (the last time there was a competitive primary in Wisconsin), so I can’t do that math, but maybe someone else could.

But we can use the updated “vampire-adjusted” weights and a reasonable guess at the vote flows in the last week (2:1 to Crowley) and plug into our non-response program (the MUBP — see the original post for an explanation).

The best guess from the authors of the paper I’m basing all this math on is that a reasonable starting point for the impact of non-response is setting phi = 0.5. And you can see why: in the L2Cv weights (which account for ideology), pushing that non-response adjustment up to 100% would have yielded a poll with 20% Hong and 60% Crowley! That’s very bad in the opposite direction of most published polls in Wisconsin.

And to be clear, since the L2Cv weights (solid line) already balance the sample by ideology, the non-response here is something about voting for Hong and Crowley that isn’t proxied by ideological identity in a survey. So you can’t use those trends to say anything about non-response among progressives. That is already taken into account by the level-shift in Hong and Crowley vote at phi = 0.

If, however, you use the L2Bv weights with the non-response dial at the 50% position, you get somewhere around 42% Hong and 40% Crowley. Given they each won about 40% of the vote, that’s a pretty good guess. The residuals here point to about a 5-6 point boost for Hong from non-response.

But again, caution is warranted, since the data for this survey all came from a week before the election. If we had data on election day, the impact of a “non-response” adjustment would be lower. (Technically, I guess, the poll not accounting for future movement in the race is “non-sampling” error, in that it’s error that comes from something other than sampling, but that is different from non-response.)

Share

Polling primaries is very hard

In summary, after adjusting my original analysis for the existence of vampires in Wisconsin, the miss in the original State Navigate polls (+20 points of rHong) breaks down roughly as a result of…

1. 10 points from misspecified age weights

2. At least 5 points of late movement (5 is what we can detect with the recontact

3. About 5 points of non-response

Adjusting for the misspecified age composition gets you within the 13 percentage point average error for the margin between two candidates in primary polls. That would indicate the primary polls and Wisconsin were not any worse than average, after adjusting for what looks like maybe a simple mistake or extra extrapolation from Michigan on the part of State Navigate.

But you can eliminate most of the error in 1 and 3 by weighting your primary poll to a reasonable benchmark for ideological composition. We don’t know what the precise ideological composition of any group of voters is, so see Recommendation 3 in my original analysis. Time for some sensitivity analysis, baby!!

If I were to editorialize a bit, I would say 5 points of ideological non-response in a primary poll is really not bad, all things considered. Especially if it doesn’t impact general election polls or what we know about public opinion writ large.

That’s just to say there will be no standalone Friday column this week. Instead, I’m sharing some graphs that got left out of my Tuesday piece on the anti-system vs left-right divide within the Democratic Party right now.

These charts do a good bit of myth-busting about the role age is playing in the party’s ideological divide right now. Reading news coverage of the recent wins for Abdul El-Sayed in Michigan or DSA candidates in New York and Denver, you’d get the impression that only young Democrats support this new wing of the party. El-Sayed’s win in particular generated headlines crediting his young base for his victory. Look out, it’s a youthhhhhhquaaaaaaake!

But if you look at ratings for key left-leaning figures in our June Strength In Numbers/Verasight poll, age is not so predictive after all. While favorability rating is not the same as primary vote choice, it tells us a lot about the direction of the party and appetite for new, fresh, urgent thinking.

This is post is free to read, but if you want to support my independent data-driven political journalism and get premium posts like these sent to your inbox 2x a week, become a paying subscriber to Strength In Numbers today. Members also get early access to data products, a private community on Discord for our political data nerds, and an ad-free newsletter experience.

Upgrade to support Strength In Numbers

If a paid subscription is out of reach, the next best thing you can do is to share this article with a friend. Organic recommendation is the #1 driver of new readers here at Strength In Numbers.

Share

Older Democrats like Mamdani almost as much as young Democrats

According to our survey, older Democrats rate Zohran Mamdani at about 72 on the 0-100 “feeling thermometer” scale.

Young Democrats are at 76 — a difference the poll can’t distinguish from zero.

Young Democrats, however, are sharply more critical of their party. Mamdani significantly out-polls the party brand and its congressional leadership among young respondents — not because under-30s (a category I sadly recently passed out of) rate lefty Dems better, but because they rate the “establishment” worse:

And it’s not just Mamdani. Young Democrats rate the party’s other left-wing figures about the same as older Democrats do — but they rate the party’s current management much colder. Schumer (34) and Jeffries (44) are the only figures under-30 Democrats put below the midpoint of the scale.

Finally, a point about absolute differences. The age gradient is strong. But if you compare feeling thermometer ratings for Mamdani and AOC vs Schumer and Jeffries, not one group of Democratic voters prefers the leadership:

This may sound pedantic, but there is a big difference between “only young people like left candidates” and “young people are voting left because they are more likely to hate the Democratic Party.” Remember, among self-described moderate and conservative Democrats, Bernie Sanders rates higher than any other current party figure we tested — Obama excepted:

The generational split in the Democratic electorate isn’t really about who likes Mamdani or AOC — Democrats of most ages do, more or less. It’s about who’s fed up with the leaders currently running the party. And that’s an anti-system thing, as much as or more than it is an ideology thing.

If you get value out of data-driven coverage of polling and public opinion, the best way to support this work is with a paid subscription to this newsletter.

Subscribe now

Related Articles

Subscribe now

Here are the big takeaways:

• Hong’s miss is a 90th-percentile polling miss, not proof the industry is broken. The pre-election average had her up about 20 on David Crowley, and she lost by one. In the Senate and governor primaries Elliott and Mary Radcliffe analyzed going back a decade, the average poll misses the margin between the top two by 13 points, and misses of 20-plus happen about 10% of the time. What’s genuinely unusual is the direction: in all eight of the 20-point misses in the database, the polls underestimated the leader — Tammy Duckworth by 20 in the 2016 Illinois Senate primary, Sam Brown in Nevada in 2024 — and the previous record for overestimating an eventual loser was 19 points, in the 2024 Puerto Rico governor’s race. (Lamar Alexander in 2014 and Spencer Cox were both overestimated by 15-plus and won anyway.) Evergreen reminder: polling primaries is hard.

• We should judge candidates by their absolute performance, not performance relative to the polls. If nobody had polled Wisconsin, the only available take Tuesday night would have been “wow, that was close.” Instead, polls set up a prior for everyone that Hong would win big — so when she didn’t, the election gets cast as a “disappointment for the left.” David argues for rooting our grading of candidates by their actual performance. Elliott says you could also use the fundamentals. But be consistent: you can throw out the polls as junk and trash them when they’re wrong, but then you can’t say candidates “underperformed” based on polls.

• At a D+7 national environment, Wisconsin and Minnesota are at worst lean D, and maybe likely. With a generic ballot running D+7 to D+8, Minnesota a couple points bluer than the country and Wisconsin about a point redder, a win by under five points for either Democratic candidate would be a surprise. Especially considering in Wisconsin the Republican nominee for governor, Tom Tiffany, is a Freedom Caucus member, election denier who voted against recognizing Biden’s win days after January 6, one of only 18 House members to oppose the resolution condemning QAnon, and very far right on abortion. Democrats will spend heavily making all of that central to their campaign — while Trump has, at best, a 40-42% approval rating in the state. Maybe this is bluer than people think?

Thanks to everyone who joined us for the weekly live stream on Substack, and we’ll be back next week!

As a reminder, paid subscribers to Strength In Numbers get to participate in our live Q&A!

If you missed our video livestream, you can watch it by clicking play on the web version of this post at gelliottmorris.com.

You can also subscribe to us on your favorite podcast app to listen on your own time. And if you do listen via one of those apps, please drop us a five-star rating and review if you feel we’ve earned it — it really helps people discover the show!

If you value this work and want to help keep it going, please consider becoming a premium subscriber to Strength In Numbers. Paid subscriptions support this podcast, the newsletter, and the time it takes to do original data-driven analysis of politics and elections. Paying subscribers also gain the ability to send in questions during our live streams, so you can directly shape the conversations we’re having on the podcast.

Upgrade to support Strength In Numbers

Share

You can read the transcript of our conversation by clicking the headline of this article to take you to the web version of the podcast, then clicking the button just below the byline that looks either like a piece of paper or is labeled “Transcript,” like so:

If you’re a reader of Strength in Numbers and haven’t yet subscribed to David’s newsletter, head to the-downballot.com.

Subscribe to The Downballot

And if you’re coming from David’s site, subscribe here to get the numbers behind the news.

Subscribe to Strength In Numbers

For a follow-up on this follow-up, see this updated analysis: KILL THE VAMPIRES.

The average error of a Senate or governor primary election poll is about 13 points. This is because pollsters have fewer benchmarks for variables that correlate with vote choice in a primary, and because the race can change very quickly late.

As of now, no pollster who was in the field in Wisconsin has published a full retrospective of their numbers based on a reanalysis of their data. In this article, I’m going to do just that.

Last year, a non-profit called State Navigate contracted me to develop a flexible survey weighting program that they could use on any poll of a statewide or state legislative district primary or general election. This program let them adjust the data they got back in their polls to be representative of the population they were trying to sample.

I did not ask for money from State Navigate, but instead to have access to the raw data and to be able to analyze it publicly for posts like these. State Navigate has agreed to let me publish the analysis below. I did not share it with them beforehand. (For what it’s worth, State Navigate was the most accurate pollster in both the Virginia and New Jersey governor elections in 2025.)

Based on the analysis below, I conclude:

• Most of the miss in polls in Wisconsin is attributable to faulty demographic targets (too many young people). This inflated Hong’s vote margin by somewhere between 5 and 10 points.

• My best guess is that the race moved 6-10 points toward Crowley after pollsters released their final surveys.

• Non-ignorable non-response within demographic categories likely further inflated Hong’s vote margin by 2-5 points.

I would usually paywall an analysis like this, but I am releasing it for the public so readers can have better-informed expectations for the accuracy of primary polls in the future.

If you learned something from this, please consider upgrading your subscription to a paid membership to make sure I can continue publishing analysis like this in the future. Paying readers make original data journalism like this possible.

Act now and get 25% off an annual subscription — before prices go up for the election season.

Upgrade to support Strength In Numbers

Three types of errors in Wisconsin polling:

State Navigate fielded two polls of the Wisconsin Democratic Primary in late July and the first week of August. The August survey was partially a recontact survey, where respondents from the July poll were asked their vote likelihood and candidate preference again.

State Navigate’s July poll had Francesca Hong at 44% and David Crowley at 15% — a 29-point lead. Their August poll, which finished interviewing on August 6, still had Hong up 44-22. (Five days later, Crowley won, 39.8% to 39.3%.)

A 22-point error on the final published margin is not a normal polling miss, even by the standards of primaries.

One thing I can rule out immediately: this was not a data-processing problem. When I take State Navigate’s raw files and re-weight them to the actual weighting targets they shared with me, I reproduce their published toplines almost exactly (Hong 44.0 / Crowley 23.3 in August, against a published 44 / 24). Instead, the problem is mostly with what the poll weighted to.

I have identified three big problems in this pollster’s data.

1. The actual primary electorate was older and more moderate than pollsters assumed

Chaz Nuttycombe, Executive Director of State Navigate, said the demographic benchmarks for his organization’s surveys were generated to be representative of state voters by predicted primary participation. First, he used the state’s voter file to compute demographic breakdowns for the race, education, region, race by region, age, gender, and registration status of participants in past Wisconsin primaries. Then he adjusted those targets based on recent primaries in 2026 and 2025. (This adjustment looks to have increased the percent of young voters in the sample — with 9% 18-29-year-olds in the poll vs 2% in the voter file.)

Other pollsters appear to have produced their primary polls in a two-stage process. First, samples were adjusted by known demographic factors to be representative of the total pool of Wisconsin registered voters. Then, respondents were given the chance to self-select into either voting in the Democratic or Republican primary (or not at all). This means the accuracy of the polls was dependent on getting a representative sample by which party’s primary voters were opting into. It looks like the voters pollsters got, however, were too young and too liberal.

It looks like both of these processes failed to anticipate an older, more moderate primary electorate. Or, at the least, weighting to a slightly older benchmark would have improved performance.

The chart below shows the August State Navigate poll under four different sets of weights, as well as the unweighted breakdown of support for each candidate and the unofficial results of the election:

Two things jump out. First, the poll’s published weights are nearly indistinguishable from applying no weights at all. That’s not because State Navigate failed to hit their targets, but because their sample already looked like their targets on age, gender, race, education, and region — so the weights had almost nothing to do.

The problem is that the targets from State Navigate themselves described a much younger electorate than the one that shows up to a low-turnout August primary — which looks like the small, old, habitual-voting slice of the state’s registered voters.

To check the impact of weighting decisions, I pulled two different pictures of the Wisconsin electorate from the L2 voter file. One universe consisted of of Democrats and nonpartisans who have voted in at least 11% of the even-year primaries they were eligible for. The other was built by taking the L2 list of active voters who cast a ballot in at least one Democratic primary since 2016. The two universes agree with each other almost exactly, which should give us some confidence in the targets being used. Among voters with a known age, both say the primary electorate is about 52-53% seniors and 2-3% under 30.

The poll samples were nowhere near that. The July sample was 36% seniors and 17% under 30 — roughly half as old and eight times as young as the electorate it was trying to measure.

When I re-weight the polls to the L2 universes (weighting by party, age, race, sex and education, all from the voter file), the August margin error drops from 24 points to about 14. These results are shown in the table below as L2A and L2B for the two universes.

Pollster performance may have been improved further by weighting on ideology. State Navigate’s published toplines showed 17% of respondents identified as Democratic Socialists, for example, and 43% additionally called themselves Progressive, Very Progressive, or Extremely Progressive:

Even after pinning age, race, sex, education, and party to the voter file via our L2A or L2B weights, the weighted sample still self-identifies as about 38% “very liberal.” I don’t have a gold-standard benchmark for what that number should be (nobody does, which is the whole problem) but when I pin it to the ideological profile of 2016 primary voters (25% very liberal — per an exit poll, so your mileage may vary), the error on vote margin collapses.

Of course, official population benchmarks for ideology do not exist, so weighting by ideology adds some researcher degrees of freedom into the pipeline. But if one does not want to weight final estimates by ideology, they could create a set of several different poll toplines assuming different ideological makeups from past exits, primaries, etc. This at least gives readers something to bite into in understanding how assumptions in the poll impact accuracy. But it is understandable that a pollster would not want to be perceived as putting their thumb on the scale and weighting a survey on an attitudinal variable without a population benchmark.

Adding targets for income brings the error to about 3 points. The recontact portion of the August sample under that final weighting scheme actually had Crowley ahead.

Share

2. There was a lot of late movement toward Crowley

Because part of the August State Navigate poll was a recontact survey, I calculated how voters interviewed in July who said they were undecided, or supported a candidate that then dropped out, said they were voting in August. I estimate Crowley was gaining about two-thirds of a point a day in the final weeks of the race — a number that could’ve been higher on the final election weekend, and likely tipped the scale.

The recontact design is frankly the coolest thing about what State Navigate did in Wisconsin this year. The August poll re-interviewed 633 of the July respondents, which I can match across polls person-by-person using a standardized ID from the voter file. That lets us watch individual voters change their minds — which they did.

Among 633 recontacts, the Hong-Crowley margin fell about 7 points in ten days, from +33.6 in State Navigate’s July interviews to +27.1 in their August ones. And that was still five days before voting day.

Where was Crowley’s growth coming from? Basically everywhere. Mandela Barnes withdrew between the two waves, and his voters in the panel went to Crowley 25% to 19% for Hong (another 14% went to Kelda Roys). Undecideds who made up their minds broke 40% Crowley, 40% other candidates, and just 20% Hong. And nearly half of them were still undecided on August 6 — providing for a mass of anti-Hong votes to flow Crowley’s way by Tuesday, August 11.

If you remove undecided voters from the survey, you can see this consolidation at work. Hong’s weighted share was within about 3 points of her actual result. Crowley’s was 6 short — and Kelda Roys was 6.6 too high and Joel Brennan 4.1 too high. The moderate lane hadn’t finished consolidating when the poll stopped calling. It finished on Election Day.

Because we have the polling microdata, we can go one step further: What if we take all the undecideds in the August poll, and then allocate them to remaining candidates based on the vote flows of respondents who were undecided in July but had decided by August?

Warning: small sample size! Among the 633 recontacted panelists, 40 were undecided in July but had picked a candidate by August. Those respondents broke 40% for Crowley, 20% for Hong, 20% for Roys, 17.5% for Brennan, and one respondent for Hughes. Apply that same break to the 10% of the weighted August poll that was still undecided, and the race becomes a dead heat: Hong 34.4, Crowley 34.3 — a 0.1-point margin, against an actual margin of 0.5. In other words, the L2C weights plus a realistic undecided allocation reproduce the closeness of the race almost exactly.

Yet even after allocating undecideds, Both frontrunners remain about 5 points below their actual vote shares, because Roys (+7) and Brennan (+5) are still holding a lot of votes that flowed away in the final days.

3. There is mixed evidence of higher survey non-response among more moderate Democrats.

In 2016 and 2024, polls overestimated support for Democrats even if a pollster accounted for party, past vote, demographics, and geography in their samples. That is, the types of Democrats and Republicans answering polls were unrepresentative of those who turned out on election day. Or, put the other way, the Americans who refused to answer polls were more pro-Trump, conditional on every demographic and political factor, than respondents to polls.

The evidence below suggests non-responders to primary polls were pro-Hong by about 2 points on vote margin vs Crowley. (That’s not so bad for a primary.)

There are a few different ways to try to measure non-response. To start with the easier one: First, I took the August poll with State Navigate’s weights and added one more weighting variable: the election result itself. In effect, I forced the poll to “predict” the outcome perfectly, and then looked at what that did to its demographics.

Not much changed. Seniors rose from 43% to 46% of the weighted sample; self-described “very liberal” voters fell from 41% to 36%.

This suggests the non-response (and late movement) favoring Hong was within demographic cells. If the poll’s error had mostly come from interviewing the wrong demographic mix, forcing the poll to match the results of the race would have reshaped the sample dramatically. Instead, matching the result required upweighting Crowley voters within every demographic group. The older and more moderate voters who did take the poll were still more Hong-friendly than the older and more moderate voters who actually showed up.

Second, we can look at the recontacts and see if Hong’s supporters were more likely to answer the poll.

Among July’s 1,284 unique respondents, 51.2% of Hong’s supporters completed an interview in the August recontact — as did 51.2% of Crowley’s. No evidence here of Hong supporters being more likely to take surveys, at least IF you already condition on taking the first survey.

But that’s a really big if! Even if Crowley supporters were just as likely as Hong supporters to respond to a poll a second time, that doesn’t mean they were just as likely to get on the horn the first time.

For that, we need a much more sophisticated analytical strategy. (Alert, it’s about to get really nerdy.)

3b. Detecting potential non-response bias ahead of time

And thus enters the “measure of unadjusted bias for proportions,” or MUBP. In a 2019 paper published in the Journal of the Royal Statistical Society, Rebecca Andridge, Brady T. West, Rod Little and co-authors propose a method for (1) combining everything observable about a respondent — demographic, geographic, and political traits — into a single score and measuring how well it predicts the survey outcome (in this case, a vote for Hong, Crowley, etc.), and (2) projecting what the true electorate-wide number would be if people’s decision to take the poll depended not just on those observable traits, but partly on their vote itself (eg, Hong voters being more eager to answer a poll about the race). Dialing that assumption up from “not at all” to “entirely” traces out the range of results different amounts of non-response bias could produce.

The MUBP, in other words, is a stress test for non-response bias. It works in four steps:

First, I take everything we record in the poll about each respondent — their age, race, education, income, party, ideology, region, how likely they say they are to vote etc — and boil that down to one number: a score that predicts how likely they are to vote for Francesca Hong v David Crowley. This score is produced by a statistical model called a regularized elastic-net regression, which is trained in such a way as to not overfit the data. In our poll, this score is strongly (but not perfectly!) predictive — it correlates with actual vote choice at about 0.6 — meaning most but not all of what makes someone a Hong voter is visible to a pollster.

Second, we take the model that generates those scores for respondents and score the electorate we believe will show up on Election Day. Then, we can measure the difference between the score for the electorate and the survey. That gap tells us how much the poll over-recruited Hong-looking people — similar to the same exact calculations you could make by, e.g., raking weights to population marginals:

• Ignoring undecideds, respondents were 49.2% Hong (unweighted) in the August poll. Projecting the predictive model for Hong Score onto the L2C electorate, we get an implied 38.8% — so the over-recruitment of Hong-looking people is worth ~10.4 points of her share even in the most weighting-friendly reading.

• This is roughly the same correction the weights themselves deliver: the L2C weights put Hong at 36% of decided voters (32.3% of all respondents — see the first chart of this post). The small difference is model-versus-crosstab bookkeeping.

So far, we have just rediscovered survey weighting. But now we do something really cool: we can calculate what the gap between the electorate and poll says about non-responders. That depends on an assumption no one can verify, which makes this a simulation analysis or sensitivity test, rather than a prediction. These assumptions are:

• Data is missing at random: Your weighting model has adjusted for all the things that might influence response, so you’re in the clear! Don’t worry about non-response bias!

• Data is missing not at random: There is some variable that is exerting a Jupiter-scale gravitational force on selection into taking your poll. If there is unexplained variance between survey response and the Vote Score you give to your respondents, be scared.

• Everything in between 0% MNAR and 100% MNAR.

Imagine, to editorialize a bit, a big dial from “Your survey is good” to “You might be totally screwed.” At one end of the dial, people answered the poll because of visible traits that flow into the Hong/Crowley/Etc Score — young, very liberal people just answer more surveys. If that’s the story, the missing voters differ from respondents only as much as the score predicts, and good weighting can fix everything.

At the other end, people answered because of how they planned to vote — the score is only a proxy for the real differences between voters, and the missing voters are far more different than they look, and no weighting can fix it. Each point on the dial thus implies a different “true” result of a survey if you had a better proxy for selection into the sample. Sweeping the dial from left to right generates a curve of possible poll results.

That’s basically what’s happening in the chart below. Against the voter-file electorate with ideology in the weighting scheme, Hong’s curve crosses the result right at the left, non-problematic end of the non-response dial: her overstatement was ordinary, observable selection, no spooky mechanism required. That’s the dark blue line. If you use different weights, however (the other dashed and dotted blue lines), you have to assume some amount of spooky uncaptured non-response force increasing Hong voters’ likelihood of taking a survey in order to get her estimated vote share back down to what it actually was (39.3%). And under State Navigate’s own targets (the dotted line), even the maximum amount of spookiness doesn’t get her there: because their weights barely moved the sample, the curve sits at about 49% no matter what you assume about non-response.

But in terms of relative probability, it does look like Hong voters were more likely to answer the poll. Crowley’s lines (orange) only reach his actual vote share about halfway up the “you’re screwed” dial — and that’s under the friendliest electorate, the L2C targets. Under the L2B and State Navigate targets, his curve never gets there at all.

This table (for the nerds) tells us what level of non-ignorable selection bias we have to assume (column 3) to explain the misfire in the polls, after controlling for demographics. A phi (the dial) of 1 is the worst-case scenario. The authors say a phi of 0.5 is a good guess for unobserved bias in your data. You have to use a phi of 0.7 to recreate Hong’s actual vote share using the data — which is not great but not terrible.

But before you get carried away, there are two caveats to the MUBP approach. First is that (because it’s just using poll results) it doesn’t take into account late movement in the election. The method is a snapshot, so it can’t distinguish “people wouldn’t answer” from “people changed their minds after the poll ended.” Second, the amount of non-response needed to explain the election result also depends entirely on what you think the electorate will look like (which is exactly why I ran it three different ways).

Using only the weights with only demographic and political variables (L2A — eg, no ideology or income in the targets), my best guess from the MUBP is that non-response was increasing support for Hong by something around 0 to 5 points, depending on whether a pollster weighted by ideology. Graded against the actual results, that explains about a tenth of the gap between the polls and the election result (the unweighted August sample had Hong up 23.8 points among decided voters; she lost by half a point). The rest was a combination of late movement and (fixable) weighting problems.

That last finding is why I think this method belongs in pollsters’ regular toolkits. You can run it before publication to assess the potential error in your poll that could be attributed to non-response bias for a candidate, as proxied by the correlation between the variables in your weighting model and the survey response.

Share

The bottom line

It would be easy to read all this as an indictment of one pollster. I don’t think that’s the right reading. State Navigate hit their weighting targets, published their demographics, ran a genuinely valuable recontact design, and — to their great credit — handed over their raw data and let me publish whatever I found. Very few pollsters would do that.

The right reading of this is that primary polling is hard in ways that general-election polling is not. It is reassuring that most of the 20-point miss by the polls came from two problems that are either (1) fixable (weighting model) or (2) unavoidable in primaries (late movement), rather than, e.g., pervasive non-response by moderates.

The first problem is fixable with accurate voter-file targets (and bias is further reduced with some handy guesswork about ideology and income). The second is fixable by fielding later — and by everyone, poll-readers included, treating a primary poll as a snapshot with a very short shelf life.

But don’t conclude the data is useless: re-weighted with corrected targets, these polls missed by about as much as the average primary poll. And the smart recontact sample also helped indicate late movement that should have prompted readers to treat the poll as less precise in the final days of the campaign.

Recommendations

Based on all of this, I would give pollsters four recommendations:

1. Weight primary polls to voter-file data on primary voters — not to an all-registered-voters frame with a screen on top, or to targets derived from other states or sources. On State Navigate’s end, adjusting weights is common practice, but appears to have driven additional error in this election (maybe by coincidence). That is an unfortunate error.

   But many pollsters also used demographic benchmarks that made the poll sample representative of Wisconsin’s registered voters generally, and then trusted respondents to self-select into the Democratic primary at random. The sample likely skewed young, ideological, and hyper-engaged — over-representing the Hong coalition. But weighting data to targets from the L2 voter file largely fixes the observed error. Weighting choices alone were worth roughly 10 points of margin here.

2. Field into the final weekend. When the race is moving quickly, being in the field as late as possible can generate more accurate data.

3. Assess sensitivity to ideology targets — if they impact results, increase uncertainty. The results of this analysis also hint at an oversampling of progressive Wisconsinites. But because there is no official population benchmark for political ideology to weight to, pollsters don’t use it (in one specification above I use the ideological makeup of 2016 primary voters, which helps). But that doesn’t mean pollsters can’t use the variable at all; a sensitivity analysis can be done without a hard target. I find that adding ideology to the weighting scheme moved the August margin about 11 points even after weighting by demographics and political party. This can be a clue to pollsters that their results are susceptible to error from the ideological makeup of their survey. One possibility is to produce multiple estimates with various ideological benchmarks.

4. Run a non-response analysis as a routine diagnostic. Above, I also apply a newer method from academic survey research — called the “MUBP,” a diagnostic for nonresponse bias — that lets a pollster stress-test a poll for non-response before publishing it. The idea is simple (the execution is not). First, combine everything you can observe about each respondent — age, education, party, ideology, and so on — into a single score that predicts how they’ll vote. In this data, that score is strongly but not perfectly predictive (it correlates with vote choice at about 0.6): most of “who you’ll vote for” is visible in traits a pollster can see. Second, use the same score to compare the poll’s respondents against the electorate you believe will show up. If the weighted poll still looks different from that electorate on the score, there is bias left over that better weighting could fix. You can also adjust how a candidate’s response score might be correlated with taking the survey at all, which allows the pollster to simulate different poll results given different levels of non-response. If the result of the poll changes under high non-response (because your support score does not fully capture support for a candidate), that’s another opportunity for exploration — and explanation to the public.

Thanks to Chaz at State Navigate for sharing his data.

In the end, the polls in Wisconsin don’t look so bad — or at least, no worse than the average primary poll in a race with a lot of movement in the final days.

Strength In Numbers is a reader-supported publication. If you read this far, you will definitely get a lot out of the additional exclusive posts each week, and our private community caht on Discord.

Upgrade to support Strength In Numbers

Footnotes

The Wisconsin voter file is notorious for having a lot of missing and incorrect data. One of the variables that is particularly errant is the age variable.

In their methodology report State Navigate wrote:

“many voters in Wisconsin have a birth year of 1800, and we don’t believe in vampires. These generally are newer voters who skew very young; about half of voters who we polled with a birth year of 1800 say they are in the 18-29 age bracket. Thus, if you were to simply take the voter file’s birth year data, you’ll get a Democratic primary electorate that is majority 65+ and has less than 2% of voters under the age of 30. Absentee requests for the Wisconsin primary (which includes requests for Republican ballots) are 5-6% 18-34. Given that this age group has been notorious for voting late/on Election Day, we find it hard to imagine a Wisconsin Democratic primary electorate that is <8% 18-29. Maybe it is; we don’t know for sure due to the ridiculousness that is the Wisconsin voter file”

Well, I am more than a little wary of using polling data that had too many responses from young people to make any sort of predictions about what age numbers would be on the voter file, so is another opportunity for sensitivity analysis.

In the plot below, I re-raked the August survey with the under-30 target set by increasingly aggressive assumptions about how old the “vampires” are — from my baseline a priori estimate (unknown ages allocated in proportion to the known ages) up through State Navigate’s estimate (half of unknowns under 30) and past it (70%) — under both the L2C weights and the ideology-free L2B weights.

Notably, stepping up the share of voters who were <30 increases error linearly. Under the L2C weights (which account for ideology and income), the margin goes from Hong +2.4 at my baseline, to Hong +4.9 using State Navigate’s stated floor of an 8%-under-30 electorate, to Hong +6.6 if you assume half of unknowns are under-30, and Hong +8.4 at 70%.

Under the L2B weights (which don’t have ideology in the targets) the same exercise runs from Hong +13.8 up to Hong +19.7, nearly all the way back to the unweighted poll’s +23.8. The reason is simple: the young people who actually answered this survey were overwhelmingly Hong voters (likely too pro-Hong, as the non-response analysis suggests), so every additional young voter you add to the electorate model pushes the poll further from the result.

Notably, the adjusted weighted estimates I present — even at a 7-10% under-30 percentage — are still much closer to the result than the original weighted or unweighted numbers.

Whatever the true age of Wisconsin’s vampires, the topline takeaways from the analysis stands: the polls had too many progressive in them, and a large amount of late movement toward Crowley.

I also don’t see why pollsters couldn’t present sensitivity analyses like this in their reports. It would add a lot of context for consumers of their estimates.

But graded against the polls, Hong disappointed. Our average of polls at FiftyPlusOne had Hong 20 points ahead of the competition Tuesday morning, with about 20% of the electorate left undecided:

The difference between expectations and reality prompted the New York Times to call Hong’s defeat versus challenger David Crowley a “stunning loss” for the left:

So the polls are causing not just high expectations for the left, but also journalists to react to world events in weird ways — grading candidates by their performance relative to the polls, not their performance in absolute terms.

This is the second time this year that polls have overestimated support for a left-leaning candidate in a marquee race. Since the rules of punditry dictate that two is a pattern, Tom Nichols, a conservative writer for The Atlantic, raises the possibility of uniform polling bias in progressives’ favor:

For what it’s worth, this is silly on its face; the same night polls overestimated Hong, they underestimated support for progressive candidate Peggy Flanagan in Minnesota by the same amount:

But Nichols isn’t totally off base. The polls in both Wisconsin and Minnesota last night performed worse than the average primary poll.

Are primary polls just useless? Are all polls so bad that you should just ignore them? Let’s take a quick look at the data.

I would usually paywall an analysis like this, but I am releasing it for the public so readers can have better-informed expectations for the accuracy of primary polls in the future.

If you learned something from this, please consider upgrading your subscription to a paid membership to make sure I can continue publishing analysis like this in the future. Paying readers make original data journalism like this possible.

Upgrade to support Strength In Numbers

Primary election polls are off by 13 points on average — but that number hides some huge misfires

Earlier this year I wrote up a historical study of the accuracy of primary polls that I conducted with my colleague Mary Radcliffe at FiftyPlusOne. We concluded then that primary election polls for Senate and Governor elections in 2014, 2016, and 2024 (years where we had the most complete data) had an expected error of about 13 points on average in the difference between the margin for the top two candidates:

On the margin between the top two candidates — which is what we really care about in plurality races, like the Democratic primary in Texas — the numbers are even worse. The expected error on the margin between two primary candidates in a race for governor or U.S. Senate is a whopping 13.0 points. And in our historical data, polls picked the wrong winner for competitive primary races (those where the margin between candidates in polls was less than 10 points) in 2 out of every 10 elections. So both Crockett’s 3-point edge and Paxton’s 3-point advantage are well within the historical predictive margin of error for Senate primary polls.

But the average also obscures some truly enormous misses. In the crowded 2024 Nevada Republican Senate primary, polls had Sam Brown up 18 points on average over the last 3 weeks of the race; he won by 45. In the 2014 Iowa Republican Senate primary, Joni Ernst’s polling lead over her closest competitor was 16 points; she won by 38. And the 2024 Ohio Republican Senate primary polls showed a near-tie between Bernie Moreno and Matt Dolan; Moreno won by 18.

The 2024 cycle was also the least accurate of the three we studied, with an expected error of 16.1 percentage points on the margin of victory between the candidates polling in the top two. The 2014 cycle was the most accurate, at 10.8 points. Neither of these numbers should make you confident in a 3-point lead.

We made the following graph of polling averages for key primaries. Note that on average, polls are right in primaries; they just have a huge spread:

But this chart really doesn’t give you a good look at how off primary polls can be. The following histogram does, showing absolute errors in some cases of more than 30 points!

Running the math with this data, a primary candidate that is up by 10 points has a corresponding win probability in their race of only 70%. (In contrast, in general elections, that number would be well above 95%.)

A related question is how often polls miss by as much as they’ve been missing recently. Here’s a table showing the probability of primary election polls being X points off:

About 10% of the time, primary polls miss the winner’s margin of victory by about 20 points.

(All the data here is from FiftyPlusOne.)

Based on this data, I would recommend taking primary polls seriously but not literally. To put a finer point on that statement: The expectation is that primary polls will be off by about 13 points on average in the difference between the margin for the top two candidates. You should expect them to err, not to provide perfect predictions.

Share

Polling primaries is just really hard

Primary election polls are not general election polls.

As Mary and I wrote:

Part of covering public opinion polls is knowing when they are useful and when they are not. While polls give us good predictions of horse-race contests (especially presidential elections) on average, they are less predictive in primary elections.

This is in part because electoral dynamics can change quickly in primaries (candidates can drop out and endorse others), but also because primary polls are simply harder to conduct. [...] In addition, primary polls tend to have significantly more undecided respondents than general election polls. This all adds up to a lot of uncertainty about the polls ahead [in primary raced]

(Emphasis is mine.)

The primary (hah) problem with primary polls is that pollsters have fewer variables they can use to make sure their polls are representative of the electorate. The response rate to the average poll today is well below 1%, forcing pollsters to make statistical adjustments to their data to try to make it representative of the electorate. This usually means comparing the demographic makeup of a poll to the Census data in a region and giving more weight to the groups that are underrepresented in a sample, and less weight to those that are overrepresented.

These adjustments work well in a general election where the variables pollsters have are highly predictive of election outcomes. Things like race, education, income, past vote, and party identification are very good proxies for how someone will vote in an election between a Democrat and a Republican.

But, as Mary and I wrote, there are fewer highly predictive variables available to weight on in primary elections, where candidate support is more correlated with non-demographic factors like ideology, issue prioritization, beliefs, and electability, and so on.

As the quote shows, other hurdles exist, too. First, it is harder to predict who will turn out to vote in a primary. The population pollsters are trying to represent — people who turn out to vote on Election Day — is an ephemeral group, coming into existence in an instant, not able to be polled precisely ahead of or after the contest. Second, the electorate’s preferences are also formed later and based more on things like media coverage and advertisements — both of which can cause a lot of late change in the race (see: 2026 MI-Sen Democratic primary).

Measurement of preference in a primary election, in summary, is just really, really hard. Expect error, not precision.

Share

So, back to Wisconsin. Was Tuesday night a “stunning loss” for the left?

Only if your baseline was that the polls would be perfect. As this analysis shows, that is not a well-calibrated expectation.

With a proper model of primary polling data, losing while being somewhere at +10 to +20 in polls is somewhere between a 1-in-10 and 1-in-30. That’s rare, but not unthinkable. And it looks like polls actually got Hong’s support pretty close, they just underestimated how well her opponent would do — in an environment with lots of undecided voters and dropouts for other candidates that came very late in the race. This is an argument for even more caution.

The practical takeaway for the rest of the 2026 primary calendar is:

1. Expect primary polls to be off by 13 points on average. This will help you mentally prepare for misfires. A 10-point lead is roughly a 3-in-4 favorite, not a lock.

2. Expect a blowout sometimes. With misses of 20+ points striking about once in every ten races, and a dozen contested primaries left this cycle, at least one more “shocking” result is the expected outcome, not a crisis for polling.

3. Don’t build theories of ideological bias on two data points. Historically, left-leaning candidates do no better versus the polls than centrists or right-leaning candidates. As we saw on Tuesday night, the tails are fat in both directions: Polls in Wisconsin overestimated a progressive by 20 points while polls in Minnesota underestimated one by the same amount. People writing about polling error with a sample size of n=2 are doing punditry, not statistics.

Moreover, I would argue that the failure of the conventional wisdom to see a Hong loss on Tuesday as possible is both a problem of the polls and of the media’s expectations for them. This is the central argument I made in my book, Strength in Numbers: How Polls Work and Why We Need Them: polls are indispensable tools for hearing what the public actually wants, especially between elections, and they can only fulfill that duty if we treat them with the amount of uncertainty they deserve. Attaching false precision to electoral projections sets polls up for failure, which prevents the people from using them to hold leaders accountable when they run afoul of public opinion.

Primary polls are really hard to get right. As such, they come with large margins of error — even when combined. Don’t expect perfection, and you won’t be disappointed.

Share

These are the so-called “anti-system” voters I have written about before. The general gist is that there’s a group of voters out there who have very weak ideological attachments but very strong anti-system orientation. Donald Trump used this group to win the presidential elections of 2016 and 2024.

After my column last Friday, my friend Alexander, a political science PhD, texted me to say he thinks this anti-system thing “could be big and a distinct overlooked factor.” But he wondered how much of it could have just been fervent opposition to the Trump administration and to current Republican policies. That wouldn’t be a standing disposition about how American democracy works (or doesn’t) for a person, so much as their reaction to the particular person in office today.

Alexander asked me to produce two extra figures on this. First, he wanted me to look at how anti-system attitudes today predicted attitudes toward Republican politicians and political figures — do they predict support for, or opposition to, Trump? And second, to figure out if anti-system Democrats are gravitating toward lefty Democrats just because they dislike Trump — or if, as the broader theory suggests, anti-system voters are drawn to anti-system candidates regardless of their ideology or orientation to Trump.

In this week’s Deep Dive, I answer both of these questions. Then, below the paywall, I introduce a new way of looking at the electorate that combines both ideology and anti-system orientation, using the following grid chart. (The chart axis and title are intentionally clipped to keep you guessing!)

But let’s take it one question at a time. Is anti-system sentiment, in fact, distinct from anti-Trump sentiment? Let’s dive in.

Strength In Numbers is a reader-funded small business supported by paying members. Subscribers get perks including premium articles like this Deep Dive, can access our private community Discord server, and support the data collection and analysis work that powers my independent, data-driven journalism.

If you want political analysis that’s honest and data-driven, not beholden to publishers’ ideological agendas or donations from corporate Super PACs, upgrade to a paid subscription.

Upgrade to support Strength In Numbers

“Anti-system” does not just mean “anti-Trump”

First, I will acknowledge that anti-system sentiment is strongly correlated with a person’s feeling toward Donald Trump (in this case, a feeling-thermometer rating from 0 to 100). Among all adults, the correlation between the 3-way “direction of country” question and Trump’s feeling thermometer is −0.68, and it holds at −0.45 among Democrats and leaners, −0.48 among Republicans and −0.52 among pure independents.

The more anti-Trump someone is, the more anti-system they tend to be — and vice versa. That holds for Democrats and Republicans.

But note, a correlation of -0.48 and -0.68 is really not that strong. There are a lot of people out there who are both anti-system and pro-Trump, or anti-Trump and pro-system (a much smaller group, to be sure). This presents an opportunity for persuasion for anti-system candidates.

Another hint that the anti-system variable is picking up a real concept, not just anti-Trump sentiment, is that anti-system voters tend to view Trump and other “in-system” figures such as JD Vance and Elon Musk much more negatively than anti-system Republicans, like Tucker Carlson.

Among Republicans and Republican-leaning independents, the ones who say the country needs “major, disruptive changes” rate Trump 37 points colder than other Republicans do, Vance 34 colder and Musk 29 colder:

In contrast, Tucker Carlson comes in at a 6.3-point gap between pro-system and anti-system Republicans. The former Fox News host has spent the better half of 2026 criticizing the president for his war against Iran, and has distanced himself from much of the MAGA movement.

Of course, this does not give us 100% statistical confidence about which way the causal arrow between anti-Trump and anti-system is pointing, but we can at least be reasonably confident in the separateness of the anti-system axis. This is consistent with the academic literature on the subject.

Subscribe now

“Anti-system” does not just mean “progressive”

We can tease the two constructs here — anti-Trump and anti-system — apart a bit more using a regression analysis. For each Democratic political figure in the original survey, I modeled a respondent’s 0–100 “feeling thermometer” rating as a function of their age, educational attainment, race/ethnicity, income, sex, ideology, anti-system sentiment, and Trump approval. Crucially, the last three variables are entered into the same regression, so that we can distinguish the effects of being anti-system relative to a person’s evaluation of Trump.

This graph shows the results. Points to the left indicate the variable (if blue, anti-system — red, Trump approval) is predictive of viewing the given figure/party more favorably. So, for example, controlling for all key demographic and political factors, anti-system Democrats view Zohran Mamdani about 15 points more favorably than those who say the current system is doing well or merely poor (but not bad enough to need “major, disruptive changes.”)

There are things worth pointing out here. First, both anti-system sentiment and anti-Trump sentiment are independently predictive of viewing lefty Democrats favorably — and, of viewing them more favorably than centrist/party figures. Senate Minority Leader Chuck Schumer is, as ever, viewed particularly negatively among Democrats, and particularly negatively by anti-system Democrats.

Second, look at Obama’s two points. Wanting the system torn up moves his rating up by just 3.7 points, which the model cannot distinguish from zero — but disapproving of Trump moves it up by 24.2 points. Obama’s is the largest Trump coefficient of anyone in the chart. This is because both pro-system and anti-system Democrats love Obama.

Compare Obama’s dots to Mamdani’s, where anti-system sentiment moves the rating 15.6 points with Trump approval and ideology both held fixed; or to Alexandria Ocasio-Cortez’s, at 14.8, or Bernie Sanders’ +13.4

Per this data, being anti-system appears to be a distinct psychological construct from ideology or partisan loyalty. And it is predictive of certain downstream traits — like voting for Democratic Socialist candidates — that are being hotly debated today.

Of course, we can’t know for now whether being anti-system would still predict voting for Zohran Mamdani in a year with a Democratic president. If you asked this during a Democratic presidency, would the same pattern appear, or is this anti-system picking up something additionally related to views about the current party in charge? What if it’s just measuring intensity of opposition to Trump, for example?

We can’t know until Democrats control the presidency again. But if the item only tracked opposition to people in charge today, it should predict support for any anti-Trump Democrat — Barack Obama very much included. But it doesn’t. It also shouldn’t separate the lefty “insurgent” Democrats from more “establishment” Democrats, since both of them oppose Trump — and yet it separates them by better than eighteen points, after controlling for potential confounders.

To understand this even more, I devote the remaining half of this post to dividing the electorate by (1) anti-system sentiment and (2) ideological self-identification.

Subscribe now

Left-wing Democrats are winning over moderates and conservatives who oppose the system

Back in 2017, the political scientist Lee Drutman made the following graph of U.S. voters using survey data from the Voter Study Group. This chart places every person that surveyed onto a 2-dimensional grid based on the x-axis, their “economic” ideology, and, on the y-axis, their “social” ideology. Then, each person is given a color depending on how they voted:

This is one of those charts you see everywhere now. In many ways, it made the idea of “multi-dimensional” political ideology go mainstream.

I wanted a similar chart, but one that sorted Americans not into Trump vs Clinton voters, but based on whether a person feels more favorably toward lefty candidates or the Democratic Party — and one where the vertical axis isn’t “social ideology,” but anti-system orientation. With such a chart, we would be able to distinguish whether/how much the appeal of figures such as Mamdani and Ocasio-Cortez is ideological vs rooted in anti-system orientation.

The chart below puts every adult on a 2-D grid with self-reported ideology across the bottom, and the anti-system “direction of country” question up the left side. Each person then gets a color from pink to blue representing how warmly they feel toward Mamdani, Ocasio-Cortez and Sanders vs Chuck Schumer, Hakeem Jeffries and the Democratic Party itself — the party’s congressional leadership and its own brand.

Here’s that chart:

The rest of this Deep Dive is for paying subscribers. I write at least one post a week reserved for the members of this community who keep Strength In Numbers running. Upgrade to continue reading and support the time and data collection it takes to do analyses like this one.

Read more

This is a wonky episode, but one with profound implications for anyone who cares about what the public wants and how — or if — the government responds to that.

Share

Here are the big takeaways:

• One of the most famous results in modern political science looks like a statistical fluke. From 2004-2014, Martín Gilens and Ben Page took thousands of opinion polls and broke down preferences for public policy among rich, middle-income and poor Americans. Then they identified policies where the rich and poor disagreed, and calculated how how often either group got what it wanted. The problem is that middle and the affluent agreed about 94 percent of the time — so Gilens and Page set aside every policy where the two groups weren’t at least 10 points apart, which meant throwing out 80 to 90 percent of 2,000 policies. The way the scholars analyzed the remaining subset produced a statistical paradox called “Simpson’s Paradox,” which obscured an underlying relationship of responsiveness to the poor even when the rich wanted something different.

• The paper looks at income differences among the wrong groups. One misfire of Gilens’ approach is it pits the top 20% of earners ($150k) against the bottom. But, e.g., two public school teachers in Bakersfield make enough to land in the top 10 percent of American households — a comfortable income, but not the world of private fundraisers and super PACs that influence politics. Page and Bartels once ran a special survey of the genuinely wealthy precisely because a normal 1,000-person poll turns up no multimillionaires at all, and using it Enns shows that the top 10 percent’s opinions track the middle class far more closely than they track the very rich. But they didn’t look at policy responsiveness.

• The system responds to the public as a whole. When Enns measures how policy tracks overall public opinion, while accounting for the influence of organized interest groups, he finds Congress does respond to what Americans want, and at least as strongly as it responds to those groups. A separate 2017 reanalysis of the Gilens data by Branham, Soroka and Wlezien reached a similar place, finding something close to a coin flip when the middle class and the rich disagree. The unexciting answer may be the correct one: policy does loosely follow public opinion, and most groups (excluding the super-rich) want the same things.

• This matters because telling people their government ignores them, when it doesn’t, discourages participation. Enns concludes by saying Americans agree with each other far more than they realize, that policy does follow them when they do agree, and that the right reaction to seeing unrepresentative slices of the electorate or society get their way is to get more involved, not less.

For more of the methods details, see this blog post from Peter about his new paper, and the study link here.

If you value this work and want to help keep it going, please consider becoming a premium subscriber to Strength In Numbers. Paid subscriptions support this podcast, the newsletter, and the time it takes to do this kind of data-driven analysis of politics and elections. Paying subscribers also gain the ability to send in questions during our live streams, so you can directly shape the conversations we’re having on the podcast.

Upgrade to support Strength In Numbers

You can also subscribe to this podcast on your favorite podcast app to listen on your own time. And if you do listen via one of those apps, please drop us a five-star rating and review if you feel we’ve earned it — it really helps people discover the show!

A reminder that paid subscribers to Strength In Numbers get to participate in our live Q&A!

You can also read the transcript of our conversation by clicking the headline of this article to take you to the web version of the podcast, then clicking the button just below the byline that looks either like a piece of paper or is labeled “Transcript,” like so:

Subscribe to Strength In Numbers

Upgrade to support Strength In Numbers

Dear readers,

Before the roundup, I wanted to share two long interviews from this week that you may be interested in. First, a conversation with Paul Krugman about the midterms, in which I argue that people are updating their priors about November far too slowly. It looks like a big blue wave. A House majority bigger than the one Democrats won in 2018 is in reach, and full control of Congress is just past even odds.

Paul Krugman

Talking With G. Elliott Morris About the Midterms

For all my interviews and more, subscribe on YouTube…

Listen now

13 days ago · 1111 likes · 168 comments · Paul Krugman

Second, I was back on WNYC’s On the Media for a wide-ranging interview this Friday, talking about the latest polling on Trump and immigration, the Michigan primary, the new forecast at FiftyPlusOne, and what role public opinion ought to play in a democracy. The link to that is here.

And with that…

Leading off: I revisit the polling on the war in Iran. A lot of smart people still think Donald Trump is escaping public accountability for it, but the data says otherwise. He’s doing even worse than Joe Biden after botched Afghanistan withdrawal.

On deck this week: Tuesday’s Deep Dive is about why Trump’s numbers on the economy are so sour. It’s not just that conditions are poor, but also that voters see him as prioritizing the wrong issues.

Plus: Data about price anxiety in 2024, and the true costs of mass deportations.

Thank you for reading Strength In Numbers for another week!

If you’re not subscribed yet, enter your email here to get these in your inbox.

Subscribe now

1. Iran is worse for Trump than Afghanistan was for Biden

Pollster Kevin Collins argued Friday afternoon against a sentiment he keeps running into, and that we’ve dealt with here at SIN a number of times — that the public has given President Donald Trump a comparative pass on a wildly right-wing presidency. Specifically, Collins calls out the below post:

“perhaps [that] is an accurate reflection of media coverage,” Collins wrote. “But I don’t think it captures public opinion accurately, which has indeed been savaging Trump.”

He’s right, and the data is quite clear here. Collins notes that Echelon Insights, a Republican firm (I think their newest data just came out, thats why Collins used it) has found Trump 24 points underwater on foreign policy, for instance — actually below Joe Biden’s -10 at this point in his term:

Now it’s true that Biden had a worse rating for handling the Afghanistan situation that he did overall; Quinnipiac had Biden at 31% approve vs 65% disapproval on his handling of the withdrawal itself. So the eighteen-month comparison above is kinda the best case.

But if you look at polls of Trump’s handling of Iran, the public is even more bleak. According to the average I run for FiftyPlusOne (right now on the SIN data portal, moving to the polls site soon), Trump’s approval for handling Iran is -30 today. The graph below shows Trump’s approval overall, on foreign policy in general, and Iran in particular over the last 18 months:

But what’s also different about Iran is how how little support there is for the president’s goals. By and large Biden’s withdrawal from Afghanistan in 2021 was popular, it’s the execution of the withdrawal that dragged Biden down. Pew found 54% of Americans said leaving Afghanistan was the right callagainst 42% who said it was wrong, even as roughly seven in ten rated the administration’s handling of the exit only fair or poor.

Iran is not an analogous case. In a poll back in March, Pew found 59% of Americans said striking Iran was the wrong decision, vs 38% who said it was right. Just 37% approved of Trump’s handling of the conflict; and 61% disapproved. Unlike for Biden, where the public was aligned with his goals, Trump’s in the position of having both his policy and execution disapproved of simultaneously.

And it has gotten worse for Trump since March. In July, Quinnipiac pegged Trump’s approval on Iran at 28% against 66% disapproval, and foreign policy at 34% against 62%. His overall job approval in that poll was 32% — the lowest Quinnipiac has ever measured for him.

Update your priors about Trump

Back in September 2025, after ABC/Disney pulled Jimmy Kimmel’s show off the air when FCC Chairman Brendan Carr threatened the licenses of the stations that carried it, I made the argument that “powerful people” (network executives, CEOs, donors, celebrities) systematically overestimate how popular the president is and how much mass support there is for his agenda. From my experience I see even an overestimation of the incidence of cultural MAG thinking in the population.

Trump won about half of the votes cast in 2024. That is only about a third of American adults. Accounting for defections and disaffection, maybe 15-20% of the population is fully on board with super MAGA stuff like shutting down ABC-owned news affiliates.

After an outpouring of support for Kimmel, he was back on the air in six days. ABC/Disney, to its credit, appears to have revisited Trump’s numbers. In early May, it told the FCC in a 52-page filing that the agency was violating Disney’s First Amendment rights by ordering an ABC station to relitigate a 24-year-old exemption for “The View” under the equal-time rules. Then, it filed its station license renewals under protest, calling the order behind them “unlawful, arbitrary, and unconstitutional.”

On July 29, ABC asked the FCC to throw out the petitions against its stations, pointing to more than 150,000 public comments made in favor of the new organization.

So chalk one up in the win column. A company updated its estimate of public opinion, and acted differently when it found out the public was on its side.

Still, not all commentary has caught up to the data. There is still an implicit impression I get when talking to many folks in the media and Washington that Trump is more popular than he seems, or not getting punished like Biden was for inflation and foreign policy.

That is not true. Trump’s numbers are historically bad — not for any one big thing he did wrong, but many small ones that each marginalize additional Americans. He is deeply, deeply unpopular. Update your priors accordingly!

Share

2. What Strength In Numbers published last week

Last Sunday’s roundup was about how Trump’s deportation agenda is underwater in every swing state, and why it looks like some Republicans in Congress have started to notice:

On Monday, our sister publication FiftyPlusOne launched its 2026 congressional election forecast, which puts Democrats above even odds in both chambers. Crossposted here:

Tuesday’s Deep Dive dug into the two groups — Trump’s 2024 voters and pure independents — who are dragging down the president to his current 36% approval and 61% disapproval in our average, a net -25 and a record low.

On Thursday, I argued that the confident takes about how new Democratic Senate candidate Abdul El-Sayed is a weaker general-election candidate (as usual with his brand of story) very thin on data:

Also Thursday, David Nir and I opened the hood on the election model and worked through what went wrong with the Michigan primary polls:

And on Friday, new numbers on the divide inside the Democratic coalition — anti-system voters rate Zohran Mamdani and Alexandria Ocasio-Cortez higher than mainstream Democrats even after controlling for age and ideology:

If you’re a frequent reader of Strength In Numbers, I’m confident you will get a lot of value out of a paid subscription. You’ll get access to all of my Tuesday Deep Dives, monthly polling data, our private community of election nerds on Discord, and more.

Become a paying member today

3. Even more numbers!

• he Economic Policy Institute built a calculator for what mass deportation costs you personally. The answer is $2,358 per taxpayer, or $268.9 billion across fiscal 2025 through 2029 — enough, for example, to cover SNAP for another 118 million recipients over the rest of Trump’s term. Trump deportations cost taxpayers $2,358 each

• Ruth Igielnik explains why Michigan’s primary polls were so far off El-Sayed’s actual one-point win. Most surveys assumed independents would be 30-40% of a Democratic primary electorate, and only one poll in the final month came from a select pollster. Why Polling Overestimated Abdul El-Sayed’s Advantage in Michigan

• Pew has a piece out charting Trump’s approval alongside every modern president in their first term. He’s last, 3 points below Biden in 2022 and 6 below his own first-term number. Republican approval is cracking too, down to 69% from 84% in February 2025. Trump’s job approval rating is low by historical standards

• Mary Radcliffe and Cooper Burton write about how Americans are more anxious about prices than they were in 2024

Share

And that’s it for this week! Thanks for reading. Strength In Numbers will be back in your inbox on Tuesday.

Got more for next week? Email your links or add to the comments below! Comments are now UNLOCKED on the Sunday roundup.

Leave a comment

Next Tuesday, the left has another shot at a big win in the Wisconsin Democratic primary for governor. There, Francesca Hong, a state house representative from Madison and member of the Democratic Socialists of America, leads the polls in the race. (Not all left-wingers are created equal.)

What politicians such as El-Sayed, Hong, Zohran Mamdani in New York City, and others have in common is that they’re running against the current political and economic system, advancing a theory for a new economically populist politics that threatens establishment Democrats as much as billionaires and corporations. It’s the playbook of Bernie Sanders and Alexandria Ocasio-Cortez, but supercharged with backlash to Donald Trump’s second term.

Look at the polls, and you’ll find this anti-system sentiment is pervasive on the political left, and highly correlated with support for the left-wing candidates winning Democratic primaries across the country this year. That's this week’s Friday column.

This is post is free to read, but if you want to support my independent data-driven political journalism and get premium posts like these sent to your inbox 2x a week, become a paying subscriber to Strength In Numbers today. Members also get early access to data products, a private community on Discord for our political data nerds, and an ad-free newsletter experience.

Upgrade to support Strength In Numbers

If a paid subscription is out of reach, the next best thing you can do is to share this article with a friend. Organic recommendation is the #1 driver of new readers here at Strength In Numbers.

Share

Every month for our Strength In Numbers/Verasight poll, we ask the same question about voters’ evaluation of the current direction of the country. Instead of the typical responses of “good track” and “bad track,” we offer respondents these options:

• Things are going well in America and no major changes are needed

• Things could be going better

• Things are going poorly, and major, disruptive changes are needed

• Not sure

Pooling all seven of our 2026 polls together (11,260 adults), we can break down the “major, disruptive changes” bloc by party identity and self-described ideology. Doing so reveals stark differences by both party ID and ideology in the sentiment that the political system needs to be remade:

Among Democrats, 90.2% of the people who call themselves “very liberal” say the country needs “major, disruptive changes” to get back on track. That percentage declines as you move right on the ideological scale: 83% of self-ID’d liberals agree, as do 77% of moderates and 67.0% of conservatives.

The relationship between ideology and “tear down the system” mindset is just as high for independents who don’t lean toward either political party, but agreement takes a uniform step down. The comparative percentages for independents run from 68% agreement at the very left down to 37.9% at the right. Among Republicans, agreement peaks at 33% among liberals, holds at 30.9% among moderates, then falls off a cliff to 17% and 13% among conservative/very conservative voters.

The partisan differences here indicate many Americans see this question as one of partisan loyalty, or perhaps of loyalty to Donald Trump. But the fact that the ideological curve persists within party reveals a residual connection with anti-system thinking.

Share

Anti-system voters favor progressive figures

The chart above is important because of what it implies about the current infighting in the Democratic Party. The “tear down the system” sentiment is a key part of the DSA appeal to normie voters.

We can observe, statistically, that this anti-system sentiment is predictive of supporting left-leaning candidates, even controlling for party, ideology, age, and other variables.

Our June poll of 2,087 adults, fielded June 17-22, asked respondents both the above question about the direction of the country and to rate 19 people/party groups on a “feeling thermometer” scale running from 0-100 (0 = cold, you hate the person; 100 = warm, you loooove them).

I use this data to compare how Democrats rate different political figures based on how they’re feeling about the direction the country is headed. The chart below visualizes the gap in feelings toward each figure between Democrats who say the country needs major changes and those who say either that “things are going well in America and no major changes are needed” or that simply “things could be going better.”

The way you read this chart is that as a dot moves to the right, evaluations for that figure are higher (”warmer”) among Democrats who want “major changes” than other Democrats. So, for example, wanting the system torn up makes a Democrat warmer toward Zohran Mamdani by 24 points. Those anti-system voters also rate Alexandria Ocasio-Cortez warmer by 23 points, Pete Buttigieg by 21, Bernie Sanders by 21, Gavin Newsom by 17, Jon Ossoff by 16, Hakeem Jeffries by 14, Kamala Harris by 12 and Barack Obama by 12. The Democratic Party itself gets a 7-point boost among anti-system voters. Chuck Schumer scores a +0.3 points, within the margin of error of zero.

Two things to note here: First, there is a general skew among anti-system voters toward Democratic politicians today. This is probably part of the general anti-incumbent — that is to say, anti-Republican — mindset in politics today. We’re in a wave election where Democrats are very likely to win the House and might win the Senate. It’s no surprise that voters who think things aren’t going well like Democrats, on average.

I ran the same comparison on Republicans, and there the “direction of country” question turns people against Trump and other GOP leaders:. Republicans who want disruptive change rate Trump 37 points colder than other Republicans do, Vance 34 colder and the Republican Party itself 26 points cooler. The only Republican-ish figure who is not rated more negatively by anti-system voters is Tucker Carlson.

Yet there is a clear tendency to additionally rate highly those left-leaning Democrats who are running campaigns for populist economic change higher than other Democrats. Anti-system voters rate Mamdani and AOC about 10 points higher than Barack Obama and the Democratic Party.

Share

Yes, I ran a regression on this

The obvious objection to the above analysis is that it’s just another way of segmenting voters by ideology. By looking at Democrats and grouping by anti-system orientation, haven’t I just selected a bunch of people who call themselves progressives, as shown in the first chart? That’s totally fair, but to this objector I say: we can control for that!

I ran a linear regression predicting feeling thermometer ratings for Zohran Mamdani as a function of (1) someone’s political party, (2) their self-described ideology, (3) their age, and (4) their anti-system orientation (whether they said the country needs “major, disruptive changes”).

Among the adults who rated Zohran Mamdani on a 0-100 scale, being anti-system alone is worth 45 points. If you then add political party to the regression (as per the chart above), that coefficient falls to +21. If you add controls for ideology, it falls further to 17, where it stays even controlling for age. (For the nerds: This effect is significant at the p=0.001 level).

Among Democrats specifically, age does nothing at all once you know someone’s ideology and whether they want the system torn up.

While this is not a causal analysis — I can’t say that becoming anti-system makes you a bigger fan of Mamdani — I can tell you that being anti-system is additionally predictive of liking left-wing candidates. In other words, even among very liberal Democrats, being anti-system is still predictive of supporting figures like Zohran Mamdani.

I can’t honestly tell you this is the single reason why Abdul El-Sayed won his primary Tuesday night. It’s one national survey with no direct vote-choice question, and even with a regression model, we aren’t evaluating the counterfactual.

But what I can say is that the rise of left-wing candidates in the U.S. over the last year seems associated with the increasing frequency with which Americans express anti-system attitudes. Given the trajectory of the country’s political system (away from, not toward, the ability to solve problems), figuring out how to campaign on that — regardless of ideology — is likely worth a premium.

Our August Strength In Numbers/Verasight poll goes into the field on Wednesday, and we’ll be asking Democrats and Republicans even more questions about how they feel about their leaders and what they want their parties to stand for. Particularly among Democrats, we’ll be asking directly whether voters think their leaders are doing enough to oppose Trump, and how important that is to their vote.

If you get value out of data-driven coverage of polling and public opinion, the best way to support this work is with a paid subscription to this newsletter.

Subscribe now

Related Articles

Subscribe now

Here are the big takeaways:

• Michigan’s primary polls missed by 12 points on Tuesday night. That is, in fact, better than the typical primary poll. The pre-election average had Abdul El-Sayed up about 13, but he barely skated by with just a 1-point win. When Elliott looked at Senate and governor primaries going back to 2014 earlier this year, he found the average poll was off by 13 points on the margin between the top two candidates—and about 20% of the time the candidate trailing in the polls won outright. Evergreen reminder: Polling primaries is hard!

  • The likely culprit in Michigan is either late movement (the polls were stale by primary day) or differential non-response bias, not just bad demographic assumptions. One pollster who was in the field in Michigan described to me a “death by a thousand cuts” rather than one big error as the culprit. It’s possible the rural, lower-income white voters he did reach may simply have been too El-Sayed-friendly, and his Black respondents broke 65-35 for Stevens when the real split may have been closer to 70-30 or 75-25.

  • The deeper problem is that primaries have fewer benchmarks of vote choice to weight to than general elections. That’s a structural problem with surveying primaries, and readers should expect more errors in the future.

• The electability debate about El-Sayed is really stupid. Across public polling of Mike Rogers against either El-Sayed or Stevens, the gap between the two is statistically indistinguishable from zero. The only way to believe Stevens was clearly stronger is to weight a single Glengariff Group survey — which has also shown the strongest numbers for Rogers of any pollster — more than everything else. That notwithstanding, when we run all the data through our forecasting model, El-Sayed holds Michigan 64% of the time with Democrats taking the Senate 55% of the time; Stevens, performing similarly, comes in at 69% for Michigan and 56% to win the chamber. Mind the uncertainty, folks.

• Democrats are in better shape now than they were in 2018. The 50+1 forecast gives them an 87% chance in the House and a median of 231 seats — four short of what they actually won in the 2018 “blue wave” — but the backtested model gave Democrats only around a 70% chance to win at this same point in that cycle. That’s because there are fewer “Lean Dem” seats the party might lose this go-around, plus district-level polling, fundamentals, and race ratings are looking more favorable for Democrats. The Senate is much closer, at 54%, with 51 Democratic seats the single most likely outcome and a 50-50 tie the runner-up.

Thanks to everyone who joined us for the weekly live stream on Substack, and we’ll be back next week!

As a reminder, paid subscribers to Strength In Numbers get to participate in our live Q&A!

If you missed our video livestream, you can watch it by clicking play on the web version of this post at gelliottmorris.com.

You can also subscribe to us on your favorite podcast app to listen on your own time. And if you do listen via one of those apps, please drop us a five-star rating and review if you feel we’ve earned it — it really helps people discover the show!

If you value this work and want to help keep it going, please consider becoming a premium subscriber to Strength In Numbers. Paid subscriptions support this podcast, the newsletter, and the time it takes to do original data-driven analysis of politics and elections. Paying subscribers also gain the ability to send in questions during our live streams, so you can directly shape the conversations we’re having on the podcast.

Upgrade to support Strength In Numbers

You can read the transcript of our conversation by clicking the headline of this article to take you to the web version of the podcast, then clicking the button just below the byline that looks either like a piece of paper or is labeled “Transcript,” like so:

If you’re a reader of Strength in Numbers and haven’t yet subscribed to David’s newsletter, head to the-downballot.com.

Subscribe to The Downballot

And if you’re coming from David’s site, subscribe here to get the numbers behind the news.

Subscribe to Strength In Numbers

Watch podcast live at 2PM Eastern

Former Wayne County health director Abdul El-Sayed won the Democratic primary for Michigan’s open Senate seat on Tuesday night, defeating Rep. Haley Stevens by roughly 1 percentage point. The result was closer than pollsters projected — which some centrist, pro-Stevens Democrats cheered on social media as evidence of El-Sayed’s lack of general-election appeal.

In recent weeks, the Stevens campaign and its allies spent much of their time and money arguing El-Sayed is too progressive to win Michigan in November, and that Stevens would have been a more “electable” candidate. The New York Times reported the day after the primary that El-Sayed’s “narrow win” sets up a “critical test” for the “insurgent” left: Can progressives win in swing states?

I find these debates about electability tiring and, usually, damningly devoid of data. The reality is if you look at the data — the polls and the “fundamentals,” to use the parlance of election forecasters — the two candidates look like they would have done roughly the same this November.

What the polls actually say about El-Sayed and Stevens

This is a bonus post, way above and beyond my typical writing output for the week, and as such is being put behind the paywall as a special exclusive for paying subscribers of Strength In Numbers. The analysis in this post took a lot of time and effort to write on a very busy day (and on little sleep, because Wayne County counts slow and my eight-week-old wakes up at 4:00 AM), and paying members of the site make it possible for me to do this. If you want independent, data-driven coverage of politics and elections, subscribe and support Strength In Numbers today.

Read more

If you want political analysis that’s honest and data-driven, not beholden to publishers’ ideological agendas or donations from corporate PACs, upgrade to a paid subscription.

Upgrade to support Strength In Numbers

Donald Trump is historically unpopular — and he knows it.

Here is the proof for the first statement. Our average of polls over at FiftyPlusOne has him at roughly 36% approve and 61% disapprove — a net -25 — among all adults, and he’s rapidly losing ground. The president set a new record low six days in a row just in the last week:

Among high-quality pollsters that released new surveys in the last week, the president’s best approval percentage among U.S. adults has been 35%. This puts POTUS lower than where every other modern president was at this point in either their first or second term — with the exception of Richard Nixon, who resigned this month in 1974. At -25, Trump’s numbers are even worse than George W. Bush’s were at this point in 2006, with the aftermath of Katrina and ongoing scandals and strains from the Iraq war weighing him down.

As for statement two, consider Trump’s tweets over the past few days. On August 1, he posted “MY REAL POLL NUMBERS ARE THE HIGHEST THEY HAVE EVER BEEN.” Technically true if he means the “disapprove” number.

Then, on August 3, he ranted even more about the “RADICAL LEFT LUNATICS” rigging the polls:

This reads like someone who knows he’s in trouble, but can’t admit it to himself. The lady doth protest too much, methinks.

But Trump’s incredibly low rating — barely above 1/3rd of the public — raises an important statistical question. With modern levels of political polarization, it should be nearly impossible for the president’s rating to be this low. Right? About 40% of people are Republicans, and aren’t the uniformly MAGA?

The answer is that two important groups of former supporters are turning on him — and a third is incredibly activated against the GOP, too.

The data in this week’s subscriber Deep Dive show how Trump’s own voters are now less loyal to him than Democrats are hostile to him. I show this is the case for three different measures: Trump approval, the generic congressional ballot, and trust on the issues each voter says matters most to them personally.

The rest of this Deep Dive is for paying subscribers. Below, I use fourteen months of Strength In Numbers/Verasight polling to identify the two groups driving Trump’s decline the most and explain why the pattern is especially dangerous for Republicans in a midterm. Upgrade to continue reading and support the time and data collection it takes to do analyses like this one.

The two groups of voters dragging Trump’s approval down to historic lows

Read more

As of 6:00 AM Eastern on August 3, 2026, we give the Democrats an 85% chance to win the majority in the U.S. House of Representatives and a 55% chance to win control of the Senate (which we define as 51 seats or more, unless an independent wins in Nebraska, Montana, Idaho, or South Dakota — unlikely, but not impossible).

Because our forecast simulates elections for both chambers in one unified model, we can also calculate the probabilities of either party controlling both chambers after Nov. Democrats win control of both the House and Senate in just over 50% of simulations, whereas Republicans prevail in less than 12%.

You can read more about the initial results of our forecast in my model launch post over at 50+1. The forecast itself lives here, and here’s an extremely detailed methodology write-up of how it works.

This project is the culmination of months of combined effort by Katie Marriner, head of visualization and engineering; Mary Radcliffe, head of research and singular in her knowledge of the polling landscape; and myself (I do the math and, nominally, direct our editorial output).

Our (50+1’s) coverage of the forecast is mostly going to stay over there; the general idea is that anything derived from the team’s efforts stays there where it benefits the whole team, as long as the tone is a match for 50+1. I make isolated exception to this when a post is better suited for Strength In Numbers (to keep things above board, SIN even pays the fair-market rate for a reusable data license from 50+1).

But I did want to take some space on the newsletter today to write a few more personal words about this project and what it means to me. I think our forecast is special compared to the competition for a few reasons.

Share

First, we have custom charts that work with fast, bespoke web design. In a world of Datawrapper charts and vibe-coded Substack embeds, Katie has done a remarkable job producing an original, functional interactive website for the forecast — and she has done it all on her own. (The rest of the site, with the polls tables and averages, is also Katie’s handiwork.) Unless you’ve built a data interactive like this you cannot comprehend how much effort it is, so Katie deserves a lot of credit for not just getting things working but also making them look nice. We’ll be adding more great charts from Katie over the course of the election.

Second, speed. One of the things 50+1 promises readers and data subscribers is pure, raw speed. Mary enters pretty much all the data that flows through our website, except for election season when we hire help and she “only” enters 80–90%, and is unmatched in her ability to get complete structured data out into the world. Our forecasting system is able to update in close to real time with this data operation Mary has built. It’s so fast and comprehensive that multiple competitor outlets, not to mention political and financial clients, now rely directly on 50+1’s data streams for their operations. (Others watch our site’s HTML for changes then copy us without giving us credit.)

Third, the model. I’ve spent the better part of the last 12 years building polling averages and election forecasting systems, and this new forecast is by far the most statistically satisfying and comprehensive yet. It combines learnings from many different places — from academic political science and financial time series analysis to Bayesian modeling and traditional machine learning — to deliver a model that is well predictive on unseen data and robust to researcher specification and overfitting. Like the website, building a production, public forecasting system of this scale is also one of those challenges that you don’t fully appreciate until you’ve tried it yourself. If you want to understand more of what I mean, go read the 10,000-word methodology post on the site. (If you can believe it, it’s a lot shorter than our internal docs.)

And finally, culture. Aside from the Associated Press and New York Times, 50+1 has more former employees of FiveThirtyEight than any media outlet. Including independent contributors, there’s six of us — at a 100% hit rate! I personally think that it’s partly thanks to our shared time there that we have a hardcore culture of showing our work, treating our readers like adults, and being honest about what we don’t/can’t know (instead of selling you certainty we don’t have for clout). And while some core components of our modeling have to stay proprietary, we are open-sourcing as much as we can, including as of today the forecast’s full backtesting record — all in the name of transparency.

Plus, nobody else is doing as much as we are to keep public, transparent polling aggregation — of horse race and issues polls — alive. Just check out our polls page. Honestly a big part of our approach is that we just care more about accurately reflecting public opinion and uncertainty than pretty much anyone else. Katie, Mary and I all run 50+1 as a side project (four gals and have other full-time jobs that pay the bills. We’re literally doing this for the love of the game.

So go on and head over to 50+1 and check out our new 2026 election forecast (RIGHT NOW GO GO GO). I would start here to get your bearings on what the model predicts right now, then go to the full site to explore your favorite races, then read the methodology if you’re a real sicko. It’s a good way to burn a few cups of coffee. (I certainly drank my fair share writing it.)

Subscribe now

Upgrade to support Strength In Numbers

Dear readers,

It’s going to be a big week here at Strength In Numbers and sister publication FiftyPlusOne.news. After working in stealth mode on a few “big swing” projects for the last 3-4 months, our small-but-mighty team is releasing our first original interactive for the 2026 midterms in the next few days. Watch this space.

I’m also hard at work finalizing the questions for our August monthly poll with Verasight. If you have any questions you think would make a good piece, drop me a line. The reason we’re doing public polling through the newsletter is to ask questions that are more in tune with voters’ concerns than elites’.

In the meantime…

Leading off: A reader asked me this week what Mainers think about ICE. Because we now have the new state and congressional district MRP pipeline here at Strength In Numbers, I can answer this question — and not just for Maine, but every state in the country. I take a look at how past Strength In Numbers/Verasight polls on immigration and deportation break down at the state level.

And, on deck this week: Tuesday’s Deep Dive will be about Trump getting “squeezed from both ends” — his own voters are cooling on him while independents move toward Democrats, and Democrats are fired up against him and the GOP. This it shows up on poll questions about approval, the generic ballot, and issue trust.

Thank you for reading Strength In Numbers for another week! On with the data.

If you’re not subscribed yet, enter your email here to get these in your inbox.

Subscribe now

1. Mass deportations are unpopular in every swing state

Strength In Numbers reader (and collaborator!) Mark asked me on Friday what public opinion on Donald Trump’s immigration policy and deportation agenda is currently looking like in Maine. This is relevant for a few reasons. First, last month, federal ICE agents in Maine killed a 26-year-old man who was not a target of immigration enforcement operations. Second, there is a competitive Senate race in Maine this year. And third, U.S. Senator Susan Collins is trying to distance herself from her voting record on immigration and ICE, which we’ll see more about.

Thanks to about a year’s worth of R&D in my spare time, we now have the infrastructure here at SIN to estimate how people in each state and congressional district feel on any question asked in our monthly survey with Verasight. (We can also ask new questions and run them through the same algorithm.) Mark asked about approval of ICE in general, which we have not asked — but we did ask five other questions about immigration and deportations that I think are otherwise illustrative.

Here’s how registered voters in every state feel about ICE and Trump’s deportations agenda, across five questions:

Approval of Trump’s handling of deportations

Easy questions first. As part of our issue-approval battery, we ask Americans each month how they feel about Trump’s handling of deportations. The text of the question is:

For each of the following issues, please tell us whether you approve or disapprove of the way Donald Trump is handling… — Deportations

And respondents are asked to pick one of four options: Strongly approve, Somewhat approve, Somewhat disapprove, Strongly disapprove — or Don’t know. After removing the Don’t Knows, registered voters in July split about 10 points against Trump’s handling of deportations nationally. Here’s what approval looks like in every state:

The range of net approval on Trump’s deportations policy runs from +20 in Oklahoma to −37 in Hawaii. In total, just sixteen states are above water, along with 160 of 435 congressional districts. Trump is underwater on deportations in 34 states, including all the swing states plus Florida, Texas, Iowa, and Alaska — all states with elections for U.S. Senate this year.

Maine, the subject of this article, is at −11 on net.

Additional funding for ICE is also unpopular

We asked the following question in July 2025, in the midst of the fight in Congress over Trump’s “One Big Beautiful Bill.” At the time, I reported that the OBBBA may have been the most unpopular budget law of modern history.

Here’s the question wording:

The Republican budget increases spending on the Department of Homeland Security, which contains both the Customs and Border Protection (CBP) and the Immigration and Customs Enforcement (ICE) agencies, by $170 billion. The money is meant to fund construction of new detention facilities for immigrants, including for families, as well as the training of new CBP and ICE agents, and additional infrastructure along the U.S.-Mexico border. Do you support or oppose this additional funding for immigration and border enforcement?

Respondents could choose between Support, Oppose, or Don’t know/not sure. Here’s the national map:

Among registered voters nationally, 43% supported the 2025 OBBBA’s ICE funding provision. Support was as high as 61% in Wyoming and as low as 29% in Hawaii, and wins the majority (of people with an opinion) in just 14 states. Support for the OBBBA’s immigration funding in Maine was at 44% in our poll.

Given how the immigration opinion landscape has changed since last July, these numbers should be considered optimistic for Republicans. After the shootings in Minnesota earlier this year, and in Houston and Maine last month, support for ICE is significantly worse now.

Low opinion of how enforcement operations have been run

Now, when we move from asking questions about abstract concepts, like approval of how the president is handling an issue, to those about real events or enforcement patterns over the last year, opinion moves quickly against the president.

For example, we asked this question in March 2026:

Do you approve or disapprove of the way the Trump administration has handled domestic immigration enforcement operations {like the one in Minneapolis, Minnesota}?

The options were Approve, Disapprove, and Don’t know.

Here’s the map:

No state’s estimate reaches 50%. Maine is at 37% approve.

Deport everyone, or a path to citizenship?

In November 2025, we asked people to take a side on deportations:

Which comes closer to your view about undocumented immigrants who have lived in the U.S. for at least five years, even if neither is exactly right?

• Deport all immigrants who are in the country without legal status, regardless of how long they have lived here.

• Create a legal path to citizenship for immigrants who have lived here for at least five years and who meet certain requirements (such as never being convicted of a crime).

Here’s the map. Orange in this case is used to show states where a path to citizenship is more popular than deporting everyone here who is undocumented.

Not one state prefers deporting everyone once voters are given an option to give citizenship to non-criminal long-term immigrants.

Deporting people without a hearing

Finally, we asked this question in February 2026:

Do you think immigrants arrested by the federal government should have the right to appear before a judge before being deported, or should the government be able to deport them without a court hearing?

This question was in response to the in-the-dead-of-night removal of dozens of immigrants in defiance of a judge last year, and of the Trump administration’s expanded use of a policy called “expedited removal” to deport people here illegally without seeing a judge.

Respondents could say that people being deported should appear before a judge, or that the government should be able to deport them without a court hearing. As above, I have removed those who said “don’t know” (this is necessary to get the MRP model to run properly).

Again, the map shows the administration’s side in green. Nationally, 35% of registered voters with a view would let the government skip the hearing — whereas 65% oppose expedited removal. Maine is just left of the national average at 34% approval for deporting people without giving them a hearing.

The Trump administration is really weak on questions of due process, especially for long-term migrants.

What this means for the Maine Senate race

Mark’s question to me was in response to a thread from New Republic writer Greg Sargent, who argued on Friday that Democrats need a plan to show Maine voters how Susan Collins is being hypocritical on ICE. Sargent wrote his piece after Semafor reported that Collins “secured a commitment” from ICE chiefs that all officers will now wear body cameras during enforcement operations — something she voted against in June.

Sargent says Collins, after voting for increased ICE funding and shooting down additional enforcement guardrails, shouldn’t “have a clear field on TV to sell herself as a moderate on ICE.”

I’m not sure if Mainers will buy Collins’ abrupt pivot on the issue. But certainly the whole saga illustrates how foolish the argument from certain centrist Democrats is that the party needs to move right on immigration and “not talk about” the issue. Here, in a competitive Senate race, we have a real-world example of how Republican lawmakers are perceiving the public opinion on the immigration issue now — and they’re not acting like the people are on their side!

Sargent cites a CNN poll having Trump 22 points underwater on immigration enforcement. Our own poll is around -12, but that’s still a starkly negative number for an issue Republicans have historically led on.

Collins knows this, which is why she’s claiming credit for body cams. Sargent is right that it’s hypocritical, and I imagine Troy Jackson, the state’s new Democratic nominee for Senate, will prosecute that argument.

But the hypocrisy is the less interesting part. With Collins claiming credit for body cams, we are seeing high-profile, electorally vulnerable members of the president’s own party campaigning on reining in his signature policy. That’s because Trump’s immigration agenda is decisively unpopular, and she knows it.

Strength In Numbers has generally been ahead of the curve in seeing the pattern of, and danger in, elites overestimating Trump’s support among the American people. That has been truer on immigration than most other issues.

Share

2. What Strength In Numbers published last week

Last Sunday’s roundup looked at what it would actually take for Democrats to win over more midterm voters:

On Tuesday, an exclusive GQR poll shared first with SIN found independent Seth Bodnar beating Republican Kurt Alme 50-44 in a Montana Senate head-to-head, while a four-way field handed Alme the win at 41-27. The uncomfortable implication is that Democrats may do better in some states by not running anyone at all:

The podcast returned on Thursday, digging into what’s behind Trump’s new all-time low approval rating:

And on Friday, the Chart of the Week: gas prices fell in June, and Trump’s approval on inflation didn’t follow. His overall approval hit a new low — 36-61, a net −25. Biden learned the same lesson — voters respond to what things cost, not to the rate of change:

If you’re a frequent reader of Strength In Numbers, I’m confident you will get a lot of value out of a paid subscription. You’ll get access to all of my Tuesday Deep Dives, monthly polling data, our private community of election nerds on Discord, and more.

Become a paying member today

New at The Data Lab

One thing went up in the Data Lab this week — the corner of the site for notes on polling methods and the wonkier work I don’t send to the full email list. It’s free, and it’s a useful companion to today’s lead story about how much survey weights affect (or rather, don’t) our MRP estimates:

And for additional reference, here’s the announcement post of my revamped MRP model, with tons of charts of congressional-level public opinion, from Monday:

3. Even more numbers!

• Data for Progress finds Democrats who take AIPAC, AI, and crypto money underperform the ones who reject corporate PAC cash. AIPAC, AI, and Crypto-Funded Democrats Underperform Democrats Who Reject Corporate PAC Money

  • Related to the anti-system axis?

• Even Trump’s own voters are lukewarm on the GOP tax overhaul, which is a problem when it’s your signature accomplishment. Poll: Even Trump voters aren’t very impressed by the GOP’s tax overhaul

• The Downballot has a new method for spotting when the electoral environment shifts — it catches moves like Dobbs within about five special elections. Worth reading if you follow the specials closely. Special elections can tell us when the electoral winds are changing — faster than ever before

• Sabato’s Crystal Ball moved 13 House ratings toward Democrats. Thirteen House Rating Changes Toward Democrats

• Decision Desk HQ says James Talarico is polling better than Beto O’Rourke did in 2018

• Good graphs in this Times piece on the MAGA media coalition coming apart over Iran and the economy. The MAGA Media Machine Is Fracturing

• This is from March but I only saw it this week: It’s 3 times harder for blue states to get disaster funding under Trump

• Ariel Edwards-Levy has a good thread of findings from CNN’s latest poll, which focuses on interesting stuff that’s not the horse-race.

Share

And that’s it for this week! Thanks for reading. Strength In Numbers will be back in your inbox on Tuesday.

Got more for next week? Email your links or add to the comments below!

Leave a comment

Subscribe now

From time to time, Strength In Numbers accepts sponsors for free posts sent out to all readers (never paywalled articles). These funds help pay for things like data acquisition, subscriber churn, and staff members, including people who help with the podcast and survey research. This post contains one such sponsorship, clearly denoted after the introductory portion of the article.

After several new polls dropped this week, Donald Trump just hit the lowest approval rating of his presidency. Worse than that, he hit a new low two days in a row (and counting). Our own average over at FiftyPlusOne has the president around 36% approve, and 61% disapprove — a net score of −25.

This is a historically bad rating for a president at this point in his term. It’s even worse than Biden’s rating at this point in 2022, and worse than Trump’s in his first go-around:

Much of that drop was directly caused by the war in Iran — specifically the rise in gas prices. Here is a chart of how U.S. adults rate Trump’s handling of certain issues, again from 50+1:

This is a very bad situation for the president. He is historically unpopular and has lost ground on all the issues on which he won the 2024 election — namely prices and immigration. The Republican coalition has shrunk significantly since Trump’s inauguration 18 months ago, and the party’s position in the midterms is increasingly bleak.

And yet, the midterms are a long way away. A lot can happen in politics in three months. One question readers of this newsletter have asked me recently is whether Trump could regain his lost ground if he can get the price of gas back down. If not 2026, what about by 2028?

Back in April, I asked whether cheaper gas could rescue Trump’s approval and concluded it probably couldn’t. Now we have a direct test of this prediction. This week’s Chart of the Week looks at the data and concludes falling gas prices won’t rescue Trump — or the Republican Party.

This section contains sponsored content.

Today’s newsletter is sponsored by Ground News, a website and app I use to keep up with news from trusted sources and monitor what outlets from all sides of the political spectrum are saying on a story. Ground News gathers news articles from over 50,000 sources worldwide and shows you a measure of each outlet’s factuality, ownership, and political bias. You can also filter news by whether an outlet is owned by private equity or a corporate conglomerate — or is an independent outlet, like this Substack!

For example, here’s Ground News’ webpage for a story out yesterday about Republicans in the Senate voting down a measure that would have directed President Trump to withdraw U.S. troops from hostilities with Iran. You can see a summary of the story and links to relevant reporting.

If you’re a news junkie, consider Ground News. You can use it as a simple-yet-smart news aggregator or a sophisticated tool that helps you deal with the challenges of our modern information ecosystem. The constant news monitoring and feature filtering help me keep my own news diet attuned to facts, not spin. Today, Ground News is offering readers of Strength In Numbers 40% off their Vantage plan, which gives you a personalized feed as well as data on factuality, partisan bias, and funding information for every news outlet.

Get 40% OFF Ground News today

Gas price go up = approval number go down

Gas prices spiked from $3.00 to $4.50 on average this spring, then came back down to as low as $3.78 in early July. That was when Trump and the media were championing a negotiated “ceasefire” with Iran — that ended up ending in a week. The national average price of a gallon of gas is now $4.10.

We can look at the relationship between President Trump’s popularity and gas prices in June and see if his approval went up the same amount as it went down when gas prices surged in March-May. Spoiler alert: they did not.

For much of Trump’s second term, his approval on the cost of living has shadowed the price of gas. The chart below shows the president’s net approval on the issue — calculated by averaging over the issue approval questions pollsters publish on things like “handling prices” and “the cost of living” — alongside the national-average gas price. (The gas price series has been inverted so that up means cheaper, to match the directionality of the approval trend.)

Especially since early 2026, the relationship between these two variables has been that when gas prices surged, approval fell; and when gas prices fell, approval ticked up. The two metrics have a correlation of about −0.8.

But there is a notable asymmetry in the relationship between these variables. The chart below normalizes each trend and scales it such that the average value in March 2026 is rescaled to 100 for both values:

While gas and approval had been closely linked from March through May, when Trump’s approval fell, the June gas price ebbing did not result in a commensurate rise in approval for Trump. While gas prices (orange) recovered almost a dollar (22%) from peak, Trump’s approval (green line) ticked up a few points. And then slides right back down to where it started.

Cheaper gas didn’t buy him much, if anything at all.

Share

We’ve seen this movie before

If that pattern above looks familiar, it should. It’s the story of Joe Biden’s whole presidency.

Annual change in the consumer price index rose from 1% in January 2021 to 9% in June 2022. By the summer of that midterm election year, President Joe Biden’s approval rating had completely collapsed. That July, he bottomed out around a -20 net approval rating, the lowest point of his term. He would nearly return there in January of 2024.

But then, in 2023, inflation receded. Yet as you can see in the chart above, Biden’s approval stayed low. I’ve written about the reason before but the basic theory is that voters in 2024 were reacting to excess prices, not the rate of inflation. When inflation “cools,” prices don’t fall — they just rise more slowly. People loathe inflation precisely because they experience it as a permanent cut to their buying power, not a temporary rate that ticks up and down.

In a newly published journal article, political scientists Derek Epp and Christopher Wlezien from the University of Texas at Austin (Hook ‘em!) show that public opinion reacts much more strongly to bad economic news in good times than to equally good news in bad times. Applying that logic to presidential approval, you’d predict that approval would fall fast when prices jump, but not that it climb back when prices ease.

As both the June-August 2026 approval-gas price relationship and Joe Biden’s presidency illustrate, people tend to form their impressions of the president’s handling of the economy and hold onto it. The forecasting literature says change in the economy tends to predict subsequent change in voter behavior throughout a president’s whole term, not just toward the end.

Republicans are counting on cheap gas to bail them out this November. The president has recently heavily promoted a series of gas stations that supposedly have cheaper fuel than their surrounding pumps.

But the evidence from his presidency so far, as well as Joe Biden’s, suggests many Americans’ evaluations of Trump and his Republicans may have already become hardened.

Subscribe now

Related Articles

Thanks again to Ground News for sponsoring today’s newsletter. Get 40% off their Vantage plan today.

Ground News paid Strength In Numbers for this sponsorship, but they did not have input on any editorial content.

Subscribe now

Here are the big takeaways:

• Trump just hit another all-time approval low. In the 50+1 polling average, his disapproval is at an all-time high of 60.8% against 36.2% approval, a net of about -25. That’s even lower than the previous low the president set the day before. Across every poll out since Friday, his net approval has fallen an average of 1.8 points in the last week, with the sharpest drops among the best pollsters. CNN’s number is as bad or worse than right after January 6th.

• Gas prices explain a lot of the collapse — and cheaper gas won’t undo it. Gas prices and Trump’s inflation approval track each other almost perfectly with a short lag, but the magnitudes on the upswing doesn’t match the downswing. Based on how a small decrease in prices correlated with Trump’s approval in mid-July, we estimate a return to $2.90 gas would still leave him around -30 to -35 on prices. New Strength In Numbers/Verasight polling finds 59% of Americans have cut back on spending, and 51% blame Trump — against 12% oil and gas companies, 12% OPEC, and 8% for Biden. Evidence suggests voters’ opinions of Trump’s economy are hardening.

• Trump’s losing on immigration and deportations now. His immigration approval slid back toward its all-time low over the past two weeks, following ICE shootings in Maine and Houston, and YouGov put him at a record low on the issue this week. Since his inauguration, when the consequences of Trump’s mass deportations agenda have made news, his numbers has fallen. Elliott says Trump is now losing on both of the pillars of his 2024 presidential campaign.

• The bull case for a Democratic wave. Democrats lead the generic ballot by six points on average. But if you add the two to two-and-a-half points midterms historically break against the president’s party by November, plus the two points likely voters have run ahead of registered voters for seven months, you get D+10. That’s not far off from where the Times (D+11), CNN (+10), and Argument/Verasight (+8) put it recently.

• The Downballot’s new congressional hex map is the best answer yet to “try to impeach this.” Daniel Donner’s cartogram gives every district five hexagons and resizes states by population while keeping their shapes. The update from David covers ten states that redrew their maps for 2026. We go through an example in Tennessee.

Share

If you value this work and want to help keep it going, please consider becoming a premium subscriber to Strength In Numbers. Paid subscriptions support this podcast, the newsletter, and the time it takes to do original data-driven analysis of politics and elections. Paying subscribers also gain the ability to send in questions during our live streams, so you can directly shape the conversations we’re having on the podcast.

Upgrade to support Strength In Numbers

If you missed our video livestream, you can watch it by clicking play on the web version of this post at gelliottmorris.com.

You can also subscribe to us on your favorite podcast app to listen on your own time. And if you do listen via one of those apps, please drop us a five-star rating and review if you feel we’ve earned it — it really helps people discover the show!

A reminder that paid subscribers to Strength In Numbers get to participate in our live Q&A!

Subscribe now

You can also read the transcript of our conversation by clicking the headline of this article to take you to the web version of the podcast, then clicking the button just below the byline that looks either like a piece of paper or is labeled “Transcript,” like so:

If you’re a reader of Strength in Numbers and haven’t yet subscribed to David’s newsletter, head to the-downballot.com.

Subscribe to The Downballot

And if you’re coming from David’s site, subscribe here to get the numbers behind the news.

Subscribe to Strength In Numbers