Study Reveals Our ML Algorithms Are Significantly More Predictive Under COVID

Understanding and predicting consumer behavior accurately is the holy grail of marketing. When you can better predict consumer behavior, you can target audience segments more precisely and effectively drive up ROAS. 

But who can feel comfortable predicting consumer behavior in a pandemic – when virtually all consumer behavior has been disrupted? If you can’t be sure of your base assumptions – or even the relevance of your existing consumer data – all bets are off.

Brands and marketers need answers. That’s why we recently undertook an experiment at TrueData to discover how well we could predict consumer behavior under COVID-19 for the quick service restaurant (QSR) sector. The results were, frankly, surprising. We discovered that, for McDonald’s, our modeling is not only highly predictive pre-COVID, but even more predictive under COVID.

The methodology

Under COVID, all QSR brands are trying to drive more foot traffic and build stronger digital connections with their customers. Mobile apps have become key to facilitating deeper digital loyalty and enabling sanitary, cashless ordering. We thus set out to see how well our data could predict store visitors or mobile app installs, pre- and post-COVID. We then took population splits drawn from our TrueData database: 

  1. GPS data on users who visited a U.S. McDonald’s in the last 90 days vs. those who did not.
  2. App install data on users who had downloaded the McDonald’s app vs. those who have not.

Against these population splits, we looked at trends over time, pre- and post-COVID. We compared each population against thousands of variables across 200 million US monthly active devices and leveraged three Machine Learning (ML) algorithms to analyze the data: Bayes, Logistics, and Random Forests.

Measurable results

When the analyses were completed, we found that TrueData models have become 28%+ more accurate under COVID at predicting potential store visitors.

A closer look

We arrived at these results by performing two sets of analyses. In the first, we analyzed how well TrueData could predict if someone was going to visit a U.S. McDonald’s over the last 90 days. The Logistic algorithm proved to be 700% better than random at identifying potential store visitors by leveraging the top 3% most predictive portion of TrueData data set. This 3% audience represents millions of devices. In the second, we analyzed how well TrueData could predict if someone was going to visit a U.S. McDonald’s pre- and post-COVID.

A clear and significant improvement

The result is that our predictive power has improved under COVID by 28%. As the graph below shows, we have significantly improved our ability to identify the unique audiences that are visiting McDonald’s under COVID.

The next obvious question was why. Why are we better at predicting now during the pandemic? We postulated that it could be that a more distinct group of people is eating at McDonald’s during COVID. So we analyzed two groups of McDonald’s store visitors – pre- and post-COVID – to see how they are different.

It turned out that these pre- and post-COVID customers cohorts were pretty identical in terms of demographics and the percentage that owned the McDonald’s app. The key difference between the two groups was found in a unique combination of the 9000+ variables in the TrueData data set. The attributes analyzed included the apps installed, demographics, past locations visited, mobile device characteristics, and more. It was this cocktail of attributes –and no single attribute – that differentiated the two groups and made our analyses more predictive. The point is clear: big mobile data + an algorithmic analysis is a winning combination. Building your customer insights or targeting models on demographics or smaller sets of data may lead to missed opportunities.

Which TrueData types drove lift?

To take it one step further, we isolated our mobile types to see which ones drove the most lift for predicting foot traffic, per the chart below. We found that app install data, as a category, was the most predictive. That is to say, the apps that you have on your mobile device are better at predicting if you are going to go into a McDonald’s under COVID than the other locations you have visited in the physical world. The implications of this are extraordinary. If you are building data models off of location data alone, you may want to explore other attributes.

Below is a detailed look at the specific types of apps that were best at predicting store visits. These predictive attributes total tens of millions of devices. For example, the presence of a very specific hyper-casual game increased the odds that you would visit a McDonald’s by 1949%. Generally speaking, many of the most predictive apps did not intuitively scream “McDonald’s customers.” At this point, if you are enthusiastic about data, you should be saying “I’m loving it!” 

What it means for you

The pandemic poses significant challenges for all businesses – but it also offers new opportunities. Under COVID, we are able to offer a highly predictive solution to help QSRs better target and acquire customers and agencies (see partner portal) claim more share of wallet for their clients. Contact us to see how our model predicts mobile app installs or request a demo to see how it can work for you.