Goals in data journalism:

1. Analyze a subject that is newsy or noteworthy

• Elections, voting, specific policy votes

2. In a way that is novel or visually striking;

• Eg, use multi-level regression and poststratification to predict support for abortion rights in each state

3. Or in a way that beats the competition

• Election forecasts that are fully bayesian

Goals in social science:

Similar to the goals in data journalism!

1. Identify a phenomena

• Maybe it is a gap in the literature or a new development

2. Measure it

• Eg, with a survey

3. Explain it

• Often involves some level of modeling or prediction
• Eg, a randomized experiment or regression

1. Weighting survey data

1. Weighting survey data

1. Weighting survey data

Potential Bayesian iteration?

• Train regression model to predict voting ("target-estimation")
• Use features with large coefficients to weight the poll

Drawback?

• Of marginal utility journalistically

2. Mister-P

Answer questions like:

• What would happen if everyone in America voted?
• Which groups support Joe Biden more than Hillary Clinton?
• How can we better measure uncertainty in polling?

2. Mister-P: If everyone voted

Guiding questions

1. How many Democrats and Republicans are there?

Given data constraints, we're really asking: How many Clinton and Trump voters are there?

2. How are they distributed geographically?

The answer lets us assign Electoral College votes.

2. Mister-P: If everyone voted

Data

1. Cooperative Congressional Election Study (CCES): A survey of 64,000 Americans

Includes demographic data and 2016 vote choice for 40,000+ validated voters

2. American Community Survey (ACS): A Census Bureau survey of 175,000 Americans

Includes the same demographic data as the CCES 380,000 “cells”

2. Mister-P: If everyone voted

Method

1. Train a predictive model on CCES data

• Multi-level logistic regression
• Predict vote choice with: age, gender, race, education, region and interactions between them

2. Use the model to predict voting habits for every eligible American

Via “post-stratification” on the ACS

2. Mister-P: If everyone voted

ACS Post-stratification

1. Each "type" of person gets their own "cell":

• One cell for white men ages 18-30 without college degrees who live in the Northeast
• Another for non-white men ages 18-30 without college degrees who live in the Northeast
• etc.

2. We know how many voters in that "cell" live in each state

3. So we can say that x and y% of each "cell" vote for Clinton or Trump, then add up

• For example, a Latino female age 18-30 with a college degree in Texas is 85% likely to vote for a Democrat for president, and there's 20k of them

2. Mister-P: If everyone voted

2. Mister-P: If everyone voted

2. Mister-P: If everyone voted

2. Mister-P: Polling uncertainty

Souce: Groves et al., 2009

2. Mister-P: Polling uncertainty

Traditional margin of error only covers one source of error (sampling)

We can use MRP to take non-response and adjustment into account too

Via the posterior predictive distribution

2. Mister-P: Polling uncertainty

In brms syntax:

2. Mister-P: Polling uncertainty

Model estimates of parameter uncertainty

2. Mister-P: Polling uncertainty

Posterior draws for every cell account for sampling, non-response, and adjustment error

2. Mister-P: Polling uncertainty

Wider error bars = truer measure of uncertainty in polling

3. Elections DLMs

Economist presidential model

1. National economic + political fundamentals

2. Decompose into state-level priors

3. Polls

Uncertainty is propagated throughout the models, incorporated via MCMC sampling in step 3.

3. Elections DLMs

It's just a trend through points...

3. Elections DLMs

(...but with some fancy extra stuff)

mu_b[:,T] = cholesky_ss_cov_mu_b_T * raw_mu_b_T + mu_b_prior;  
for (i in 1:(T-1)) mu_b[:, T - i] = cholesky_ss_cov_mu_b_walk * raw_mu_b[:, T - i] + mu_b[:, T + 1 - i];
national_mu_b_average = transpose(mu_b) * state_weights;
mu_c = raw_mu_c * sigma_c;
mu_m = raw_mu_m * sigma_m;
mu_pop = raw_mu_pop * sigma_pop;
e_bias[1] = raw_e_bias[1] * sigma_e_bias;
sigma_rho = sqrt(1-square(rho_e_bias)) * sigma_e_bias;
for (t in 2:T) e_bias[t] = mu_e_bias + rho_e_bias * (e_bias[t - 1] - mu_e_bias) + raw_e_bias[t] * sigma_rho;
//*** fill pi_democrat
for (i in 1:N_state_polls){
  logit_pi_democrat_state[i] = 
    mu_b[state[i], day_state[i]] + 
    mu_c[poll_state[i]] + 
    mu_m[poll_mode_state[i]] + 
    mu_pop[poll_pop_state[i]] + 
    unadjusted_state[i] * e_bias[day_state[i]] +
    raw_measure_noise_state[i] * sigma_measure_noise_state + 
    polling_bias[state[i]];
}

3. Elections DLMs

Poll-level model

i. Latent state-level vote shares evolve as a random walk over time

• Pooling toward the state-level fundamentals more as we are further out from election day

ii. Polls are observations with measurement error that are debiased on the basis of:

• Pollster firm (so-called "house effects")
• Poll mode
• Poll population

iii. Correcting for partisan non-response

• Whether a pollster weights by party registration or past vote
• Incorporated as a residual AR process

3. Elections DLMs

Notable improvements from partisan non-responseand other weighting issues

3. Elections DLMs

Notable improvements from adjusting for partisan non-response and other weighting issues

3. Elections DLMs

2016: good!

3. Elections DLMs

2020: not as good!

3. Elections DLMs

Problem with non-response/weighting adjustments

1. Pollsters change their methods

2. Not all adjustments work

3. Elections DLMs

Solution? Conditional forecasting!

- Present aggregates assuming some amount of polling bias.

- As a way to explain to readers how bias enters the process of polling

- And what happens to forecasts if bias now does not follow historical distributions

3. Elections DLMs

Conditional forecasting:

3. Elections DLMs

2. Rerun simulations

3. Elections DLMs

2. Rerun simulations

Advantage: leaves readers with a much clearer picture of possibilities for election outcomes if past patterns of bias aren't predictive of bias now (2016, 2020)

3. Elections DLMs

But exploring parameter conditionality is not always necessary or helpful:

Thank you!

Website: gelliottmorris.com

Twitter: @gelliottmorris

Questions?

These slides were made using the xaringan package for R. They are available online at https://www.gelliottmorris.com/slides/