How analytics help campaign decision-making
Biden for President’s Director of Paid Media Analytics shares lessons learned
Political campaigns are all about choices - who to reach, where to spend money, how to spend every dollar efficiently. Sometimes, campaigners rely on qualitative insights and feedback to make those decisions, and other times, campaigns build large-scale data & analytics operations to help inform their team’s next move. For this week’s Campaigner, Arena’s Debra Cohen spoke with James Booth, who was Director of Paid Media Analytics on the Biden campaign. James shares his thoughts on effective campaign decision-making and how analytics can help provide context to what campaigns are seeing on the ground.
Q&A with James Booth on campaign analytics
Campaigner: You ran the paid media analytics program on the Biden campaign. Can you talk to me a little bit about your team and what your day-to-day was like?
James Booth: If we start with the headline objective, most political campaigns - whether it's a presidential campaign or a local council race - are trying to answer a couple of fundamental questions.. The first is, how do we stitch together our coalition of at least 50% plus one voters? And in particular, who are the voters who are gonna make a difference between winning and losing the election? Secondly, what is a message that's gonna resonate with those voters, and move them over to make a decision to vote for us? And third, where do we reach them?
A campaign needs to reach voters with volume and velocity, and paid media analytics is about plugging in data science and other quantitative work in order to contextualize the work their paid media department is doing as well as provide the context on: is what we are doing working?
The Biden campaign couldn't so easily knock on doors last year, so a lot of energy instead went into donating money to us. We had kind of more money than God (laughs), and spent somewhere close to a billion dollars on TV, digital, and mail advertising. The campaign needed to make decisions about where to spend all of that money, who to advertise to, understand if that was working, and have context for making the bigger-picture strategic decisions.
A presidential campaign is very big, so the analytics team is delivering data and insights and planning reports and evaluation reports. Really, we're creating what we might call “choice architecture” for a decision-making machine that includes thousands of people. That choice architecture is to help our campaign decide how to reach out to the right people with the right message, and do so at the right levels.
So where the rubber hits the road - the Biden Paid Media and Analytics team was trying to stitch together a distribution of reach and frequency with our media targeting and buying in a sensible way. Reach was particularly interesting to us. So we spent a lot of time doing research on TV and digital and mail consumption, thinking about how can we get in front of as many people as possible - into every nook and cranny for the voters who we wanted to make sure we were communicating with during the campaign.
Campaigner: One of the big questions often asked is: Should we spend money on TV or on digital or on mail? Are there trends you're seeing about how campaigns in this cycle should tactically approach the lessons you learned in 2020?
James Booth: In the analytics world we have sometimes approached some of these questions in an academic way, but when I hear people ask, should I do TV, or should I do mail? Do those things work? The question is, what's in your ad? What's in your mail piece? It comes back to that fundamental equation of, who are we trying to communicate with? Who do we believe is gonna make the difference between winning and losing the election and what is a message that's gonna resonate with them and where can we reach them? So it kind of depends.
Campaigner: Diving back to the shape of your team, can you talk a little bit more about what different kinds of roles are on a paid media analytics team and the skills that you were looking for when building your team??
James Booth: So I think there's probably three foundations to an analytics team generally; and if you think about your career in political analytics, as you grow, there are three core ingredients. The first is the bedrock of analytics programs, including a paid media analytics program which is engineering. We need to work in environments that require large databases, tooling, and platforms. This tends to be the smallest by headcount part of the team, but it's = a crucial ingredient. For team Biden, we had an amazing embedded engineer on the paid media analytics team. Emily Cogsdill, who was was responsible for things like setting up our architecture, setting up our pipelines, importing extremely large data sets of individual or viewership data. Number two is data science and methodology, broadly speaking. Ultimately what we are doing is analyzing and extracting insight from data. And one core part of that is, understanding the methodologies needed to solve data science problems and extract that information. The paid media analytics team was very much an applied- and programmatic-facing interface team. But data science is the second key ingredient or skill set.
And then the third is the analytics and analysis, and this is ultimately what we are doing and what our team was stacked with. We were applied analysts who were taking models, taking modeling results, taking experiments, and taking data and extracting insights in order to create recommendations and context and tools, and as I said, choice architecture for the broader campaign team, and bridge that gap between research and data and action. And, that process of analysis and generating insight and generating partnerships to solve ultimately the questions that everyone on the team is driving towards was particularly foundational for us.
Campaigner: You were talking a little bit before about this choice architecture; can you give an example?? How did you eventually narrow down all this data so it could be actionable?
James Booth: So I can give a very pedestrian technical example, and I think the more practical questions that campaigns face tend to have more precise answers. The bigger picture, strategic questions tend to be ones where what we are trying to do with analytics is create context. So one of the things we had to do was decide who was gonna be in our mail universe. We had to send hundreds and hundreds of mail universes, and there is a process of determining who's gonna receive pieces of mail. And that is a binary solvable, tactical question. It was our responsibility to come with those people, right? Like that's a very tactical, granular, example.
Somewhere in between I suppose is how much money are we gonna spend in what states and media markets on TV? And so this becomes somewhat more of a gray zone because that was heavily driven by research. We would be partnering with the data science team on research coming out of modeling on who was moving in the race and who we needed to be communicating with. And then even more importantly, which states and media markets were gonna make the difference between winning and losing? And Becca Siegel our chief analytics officer and Meg Schwenzfeier, who ran the data science team, had a really thoughtful and smart way of creating an approach to modeling that built-in uncertainty and that built in our awareness that we might be wrong about some of these things, and how do we create research that is robust and puts us in the states that are gonna make the difference between winning and losing the election with the people who are gonna make the difference between winning and losing those states, whether or not we were right, exactly right in September about where the election was. So we were creating tools and analytics on top of that research to optimize resource allocation for, say, TV spend by market.
But then there's also a strategic component to that, that is not just a data question. We were able to contextualize for those decisions. But I think it's very important, especially when you're doing analytics to understand very thoughtfully, the dividing lines between what are data decisions and what are things that are not necessarily scientifically answerable, and generating context and providing input into that and creating informational feedback loops without crowding out strategic components of those decisions as well.
And I tend to think that, I have been sitting mostly in my career for a while on the information production, production of numbers, analytics part of that process, but I spend a lot of time thinking about what is actually not an answerable, scientifically answerable question where we have to just bring context and be careful about crowding out strategic questions with scientific-ish answers. I think getting that balance right is very important.
Campaigner: You’re clearly talking about a campaign with a tremendous amount of resources, and I know you've worked on a mayor's race and races of all sizes. A lot of these smaller races have to do a lot more of that guessing or strategy context versus the data. When you worked on smaller campaigns with fewer resources, what were some of the ways you navigated these decisions?
James Booth: Even if you don't have polling, there are still some things you can do to have a decent sense of who is going to be voting, a decent sense of the makeup of the electorate. Campaigns can still think about who are our targets, what are our messages and put these things down on paper. We may know we’re not gonna be going after x group of people, but we are going to go after this group of people. Who should we be talking to with our mail pieces? Who’s responding well to this tactic?
I guess this is extremely unscientific, but good, thoughtful, local knowledge is still very valuable. Getting information about who is responding to the candidate, in a qualitative sense, on the ground is very valuable. On a local race, you don’t necessarily need to run a test to know that your message is working when people are coming off the street and telling you something is resonating.
I think of experimental results and message test results and hard data points that analytics brings to the table as kind of large pylons in the sea. You may have fewer pylons on a local campaign, but either way, you should be listening and thoughtfully trying to combine the things you do know with the context and generative insight you're getting on the ground, and chart a strategic path forward.
Campaigner: Looking back on your work for the Biden campaign, were there specific things that you learned in your experiments that you think is worth bringing into future elections?
James Booth: This is something I've thought about quite a bit. I think taking lessons from presidential campaigns is something you need to be very judicious about because a presidential campaign is unlike any other campaign in the world when you're talking about the volume of communication you can deploy, and the amount of information people already have about your candidate.
There were four years in between 2016 and 2020, during which there was an awful lot of information about Donald Trump and Joe Biden. Most of that information was not coming from either campaign. The overwhelming majority of people who voted for Hillary Clinton and Donald Trump in 2016, were still voting for the Democratic candidate or Donald Trump in 2020. So the research process of trying to understand what's moving people, who are moving in their vote choice, there is a lot more noise to signal ratio in that campaign than ever before. Though it’s getting tougher as races become more nationalized and polarized and information becomes more saturated down-ballot too.
Campaigner: You said we have a pretty good sense of who's gonna vote, but my observations over the last couple of elections is that is kind of changing. Turnout is so high and Democrats don't necessarily seem to have a good sense of who's gonna vote on the Republican side. Are there things you think we can be doing on the analytics side to have a better pulse of the electorate?
James Booth: The political realignment we are facing - not only in this country, but around the world - is turning on its head what a high turnout election means. Trump obviously blew the roof off of Republican turnout in 2020. Some of the most sporadic voters are actually kind of Trump voters, where sporadic voters used to mean more Democrats in the past. I think that has some strategic implications for the kind of questions we spend time thinking about and solving in the future.
On the broader point of knowing who's gonna turn out, I think maybe from a baseline and in particular down-ballot perspective, thinking about who the electorate is, it's relatively stable - but at the margins, a lot of things are changing in a very hard to predict way.
The way we handle that comes back to what are answerable and unanswerable questions. Forecasting the future is something we mostly do not actually try to do. We are mostly aggregating information about the current state of the world and providing context to the fundamental questions that campaigns and political parties are trying to answer both strategically and tactically.
What that means to me is that at those tactical margins we should be creating and producing information from an analytics perspective and then ultimately create strategies for communicating to voters. But we should be communicating in a way that isn't in reliance on assumptions - we should not be in a position where if a certain assumption about turnout comes out wrong, we were communicating to the wrong people. We should just never be putting ourselves in that position. Kind of being very aware of what we don't know as well as what we do know around that I think is essential.
Campaigner: My last question: If someone wants to work in paid media analytics or just data and campaigns in general, what advice would you give them?
James Booth: Two things: the first is that I think you need a technical skill set that allows you to operate in a high velocity, highly technical environment. That means you have to be able to code in SQL, whether you're an engineer or data scientist or an analyst or communicator. There are certain fundamental analysis, technical skills you need. Those tend to be developed at national consulting firms, at the DNC, at national umbrella organizations. So if you’re interested in doing analytics on a presidential campaign, you need to be developing that skillset and getting some experience in those national contexts.
I’d also advise to not get overly fixated on “the widgets” of what you're producing, right? Don’t get too caught up in the modeling techniques and the infrastructure for its own sake, and give yourself the ability to learn politically and grow politically, and understand what consumers of your information are thinking about. We talked a little bit about how I went out and managed a local campaign. It was extremely different from my background, but [my colleagues] were extremely professional, experienced consultants in that race, who I learned an enormous amount from. It gave me that big picture and getting out of DC - even just knocking doors on a state leg race is important. Build your ability to combine theory and intuition and macro-level context for how races are won so you can understand why you are doing your parts of the process.
Think judiciously about both the level of specialization you need in terms of skillsets, and also get a little local political experience to understand the broader context. I think both of those things are very important to be effective.
Arena Toolbox Highlight:
James discusses how data can be used to decide what voters to talk to and how to talk to them. Campaigns and advocacy organizations in most states can’t coordinate on strategy due to campaign finance law. That means voters are sometimes contacted multiple times by different campaigns operating in parallel efforts or not at all. The Democratic Data Exchange (DDx) was built in 2019 to address these outreach duplication challenges. Check out one of the newest tools in Arena Toolbox that explains how you can use the DDX. >>
That’s it for Campaigner this week! If you enjoyed reading this issue, give it a share on the socials! 🙏