How much did marketing activities contribute to the sales pipeline? For every dollar spent on campaigns, how much is gained in profit? Does the marketing team justify its existence? Many marketers are on a mission to find these answers, much like the “pot of gold at the end of the rainbow”. Yes, I’m talking about your marketing return on investment number (mROI).
There’s ROI, then there’s true ROI. Attribution is often not true attribution. Depending on how you go about calculating attribution, you’ll find you get a different type of ROI result.
True attribution answers the question: how much of a deal was actually marketing’s responsibility. Separating correlation from causation is one of the most frustrating things to do; however, if you are ambitious and if you leverage the richness of Pardot data properly, then it’s possible. This is our story about how we made true mROI happen.
In this post, I will cover:
- The challenge we were faced with,
- How to utilize Pardot Einstein Attribution to make marketing operation decisions,
- How to integrate Pardot data into a full-funnel measurability model,
- The statistical background and system architecture of our own model,
- The call out metrics and insights we are now able to get out of our data visualizations.
Part 1: The Marketing Attribution Challenge
Let’s take it back to the start. This is how our Pardot Lifecycle report looked like at the end of every year. The challenge I was tasked with solving is this: out of 1.6 mil visitors, I had to look down the funnel to the $1.7 bil revenue and figure out how much was a direct cause of marketing activities.
I have a background in engineering and statistical analysis before turning to marketing. I am all too familiar with one of the most frustrating things to do in marketing reporting: separating correlation from causation.
- Correlation – when A happened, B happened.
- Causation – when A happened, B happened as a result of A happening. Without A, B would never have happened.
As you can imagine, this is not a problem that you can solve due to the human biases and preconceptions we have. A human would not be able to make heads or tails of this information – you need a machine!
The way we went about it was laborious – we were, in a sense, writing our own book. We needed the help of an analyst to help us understand our data structures, create the data model, and then deploy the model.
That’s the past. I’m happy to say that if you want to recreate this now, you don’t have to go through all that pain. There is a Salesforce product that saves you most of the hassle here: Pardot Einstein Attribution.
Part 2: Pardot Einstein Attribution
We used Pardot Einstein Attribution (and were a pilot customer, playing a part in its development) and have been extremely happy with the product. These are some of the key aspects that make it particularly valuable for us:
- It allows you to move away from manual/rules-based attribution to a data-driven, hands-off model.
- Solves the huge failure point of Opportunity Contact Roles with ‘virtual OCRs’.
- Uses Shapley Values to correctly split the influence of a deal across the campaigns that influenced it.
- Plug-in-and-play (once your organization purchases the correct Pardot edition)
Everything seemed to happen at the same time – Salesforce were putting the plans together for Pardot Einstein Attribution while we were building our own models. We were able to share what each other had learned. The result was Salesforce building a better, hands-off solution that is delivering results that helps even more organizations.
How does Einstein Attribution look in action? How can we make decisions with it?
Let’s use an example of a webinar campaign we ran last year. Which opportunities did this campaign influence? And how much of the opportunity amount is attributable based on the campaign touching one or more of the decision-makers involved in the opportunity?
The granular level of detail is fantastic, right down to the euro and cent level:
One downside I found is that Einstein Attribution assumes that 100% of the credit should go to marketing, that the total opportunity amount must be split between the related campaigns in some way.
This may be the case for marketing-driven organizations, younger, fast-growing, or from certain industries (eg. SaaS); however, in a company like ours (which is 187 years old), with sales cycles that last 2-5 years and projects that are €100mil+, the efforts contributing to closing the deal definitely go beyond marketing campaigns. Not all of our revenue is sourced from marketing. A part of the revenue is, and I need to know exactly how big that part is.
Part 3: Building Our Deep Attribution Model
So, we went about building our own model that would allow us to get to that real attribution. You’ll see the model below.
If you are not interested in the architecture, feel free to skip ahead.
What we do is fetch data from our enterprise data warehouse. We have several pieces of code running in docker containers that are either going to get the data, create the model, re-evaluate the model or generate figures with key results. Objects are stored in an AWS bucket, with an API that can grab data from those objects, and either displays them on a marketing dashboard or I can query it, on-demand, to find out specific results.
First, we tell the model to grab the following datasets: opportunity, account, contact, lead, prospect, prospect activity, campaign.
As our data is relatively dirty, we have to ‘fuzzy match’ in order for the model to understand complex relationships across accounts. With the list of accounts in the screenshot below, for example, I need to tell the model that Carnival Maritime, Carnival Cruise Line, Carnival Corporation etc. are the same organization. Our org treats them as different records and each may have their own opportunities. However, in reality, if an opportunity closes for one of these accounts, I need the model to understand that the marketing touchpoints for the other ‘sister accounts’ must also be taken into consideration, there needs to be marketing attribution because the two are connected in real life.
When the ‘fuzzy match’ is done, we use something called a support vector machine. These are machine learning algorithms that are normally used for classification, but we use it for regression here. This allows the model to understand complex relationships in the data with good accuracy, not just linear ones.
Part 4: Running Key Marketing Scenarios
Now I could run some scenarios with the models. This is what I was really looking forward to doing: running the models as if there was zero marketing activity to see how the sales pipeline would have looked. Finally, I could isolate the effect of marketing.
This is my legacy so far, what I am most proud of in my working life! I have certainty that 10.3% of our net sales are directly attributable to the effect of marketing.
This also leads us to some interesting questions. Does marketing justify its existence? For example, if marketing generated €100 in sales but €100 was spent on the marketing activities, then the answer would be ‘no’. However, if we look at our organization, I calculated that for every €1 we spent on marketing, the organization gained €3.45 in profit (EBIT).
We can now visualize our whole funnel – reach, engage activation, nurturing, and through to euro revenue and the impact of each variable – in one dashboard (the one below is built using Power BI):
This is only possible if you are ambitious and if you leverage the richness of Pardot data properly (most likely through the API).
- Einstein Attribution rocks! It allows you to do things you couldn’t have imagined possible two years ago with a plug-in-and-play product.
- Leverage the Pardot API for data I/O. Once you understand the richness of Pardot data, you will be amazed at what’s possible.
- As a team, we’ve come so far – I feel like we have only just scratched the surface!