The researchers found that a range of characteristics — such as mentions of issues or facts about a candidate, whether an ad was positive or negative in tone, and aesthetic choices like whether the ad featured everyday voters or the candidate themselves — does not reliably predict which ads performed best.
The best-performing ads were more than twice as effective as an average ad, so being able to predict what will resonate with voters matters a lot. Increasing the effectiveness of an ad could be meaningful when it comes to campaigns making large ad buys — and potentially getting double the persuasion return for their money.
“If you make two ads, your best guess should be that one of them is going to be 50 percent better than the other. That’s a big deal, right?” said David Broockman, a professor of political science at the University of California, Berkeley and the corresponding author on the paper. “Or if you make four or five ads, the best one is going to be twice as good as the typical ad. That’s a really big deal.”
What makes it particularly challenging is that trends that appeared in one cycle did not always persist to the next. For example, ads that highlighted issues — broadly, any issue-focused messaging — were more effective than other ads in 2018. But in 2020, issue-focused ads in congressional and Senate races were less effective than other ads, which included spots focused on character or biography. Ads with a positive tone seemed slightly more effective in 2018 and less effective in 2020, although not by statistically significant margins in either case.
More broadly, the results from campaign experiments on Swayable can help push back on arguments that topics such as racial injustice or foreign policy don’t belong in ads, said James Slezak, the firm’s co-founder and CEO. In some cases, those ads tested well in the platform’s experiments. And evidence that a particular ad works can help campaigns convince donors who might otherwise be skeptical to chip in funds.
But the study’s authors also note that the greatest benefits would likely accrue to the best-funded campaigns and groups that can afford to create (and test) many ads in real time, not cash-strapped efforts further down the ballot.
Still, the biggest lesson for all campaigns may be to not rely too much on what worked in the past.
In 2020, some of the most effective ads tested for the Biden campaign were direct-to-camera video testimonials — featuring everyday voters shot as if they could be on Zoom, rather than slick campaign ads. But the style and substance of ads will likely be dramatically different this year, said Nate Lubin, an ad consultant on Biden’s 2020 campaign, even if the presidential race is poised to be a Biden-Trump rematch.
“It’s a different set of concerns that people have now,” Lubin said. “So I think the things that are going to be effective are very likely to be different.”
A version of this story first appeared in POLITICO Pro’s Morning Score newsletter.
Sign up for POLITICO Pro.
