How much did marketing activities contribute to the sales pipeline? For every dollar spent on campaigns, how much is gained in profit? Does the marketing team justify its existence?
Many marketers are on a mission to find these answers, much like the “pot of gold at the end of the rainbow”. Yes, I’m talking about your marketing return on investment number (mROI).
There’s ROI, then there’s true ROI. Attribution is often not true attribution. Depending on how you go about calculating attribution, you’ll find you get a different type of ROI result.
True attribution answers the question: “How much of a deal was actually marketing’s responsibility?”
Separating correlation from causation is one of the most frustrating things to do; however, if you are ambitious and if you leverage the richness of Pardot data properly, then it’s possible. This is our story about how we made true mROI happen.
In this guide, I will cover:
- The challenge we were faced with.
- How to utilize Pardot Einstein Attribution to make marketing operation decisions.
- How to integrate Pardot data into a full-funnel measurability model.
- The statistical background and system architecture of our own model.
- The call out metrics and insights we are now able to get out of our data visualizations.
Our Marketing Attribution Challenge
Let’s take it back to the start. This is how our Pardot Lifecycle report looked like at the end of every year. Which, out of 1.6 mil visitors, had influenced the $1.7 bil revenue funnel, to what extent? In other words, I was tasked with solving what revenue was a direct result of marketing activities.
My background is in engineering and statistical analysis. I am all too familiar with one of the most frustrating things to do in marketing reporting – separating correlation from causation.
- Correlation: When A happened, B happened.
- Causation: When A happened, B happened as a result of A happening. Without A, B would never have happened.
As you can imagine, this is not a problem that you can easily solve, due to the human biases and preconceptions we have. A human would not be able to make heads or tails of this information – so, you need a machine.
The way we went about it was laborios – in a sense, we were writing our own book. We needed the help of an analyst to help us understand our data structures, create the data model, and then deploy the model.
That’s the past. I’m happy to say that if you want to recreate this now, you don’t have to go through all that pain. There is a Salesforce product that saves you most of the hassle here: Pardot Einstein Attribution.
Discovering Pardot Einstein Attribution
We used Pardot Einstein Attribution (and were a pilot customer, playing a part in its development) and have been extremely happy with the product. These are some of the key aspects that make it particularly valuable for us:
- It allows you to move away from manual, rules-based attribution to data-driven, ‘hands-off’ models.
- Solves the pain point of missing Opportunity Contact Roles, by automatically creating ‘virtual OCRs’ behind the scenes.
- Uses Shapley Values to correctly split the influence of a deal across the campaigns that influenced it.
- Plug-in-and-play (providing you purchase the appropriate Pardot edition).
Luckily for us, everything seemed to happen at the same time. Salesforce were putting the plans together for Pardot Einstein Attribution while we were building our own models; therefore, we were able to share what each side had learned. The result was Salesforce building a better, hands-off solution that is delivering results that helps even more organizations.
How Does Einstein Attribution Look in Action?
Let’s use an example of a webinar campaign we ran last year. We wanted to answer the following questions: Which opportunities did this campaign influence? And how much of the opportunity amount is attributable, based on the campaign touching one (or more) of the decision-makers involved in the opportunity?
The granular level of detail is fantastic, right down to the euro and cent level:
The Downside to Einstein Attribution
One downside I found is that Einstein Attribution assumes that 100% of the credit should go to marketing – in other words, the total opportunity amount must be split between the related campaigns, in some way or another.
This may be the case for marketing-driven organizations, who are younger, fast-growing, or from certain industries (eg. SaaS). However, in a company like ours, which is 187 years old, has sales cycles that last 2-5 years, and projects that are €100mil+ – the efforts contributing to closing the deal definitely go beyond marketing campaigns. Not all of our revenue is sourced from marketing. A part of the revenue is – and I need to know exactly how big that part is.
Building Our Deep Attribution Model
So, we went about building our own model that would allow us to get to that real attribution. You’ll see the model below.
If you are not interested in the architecture, feel free to skip ahead.
We fetch data from our enterprise data warehouse. We have several pieces of code running in docker containers that are either going to fetch the data, create the model, and re-evaluate the model – ultimately, generating figures for key results.
Objects are stored in an AWS bucket, with an API that can fetch data from those objects. We either display data on a marketing dashboard, or data can be queried, on-demand, to find out specific results.
- First, we tell the model to fetch the following datasets: opportunity, account, contact, lead, prospect, prospect activity, campaign.
- We have to use ‘fuzzy match’, as the data is relatively dirty, in order for the model to understand complex relationships across accounts. With the list of accounts in the screenshot below, for example, I need to tell the model that Carnival Maritime, Carnival Cruise Line, Carnival Corporation etc. are the same organization. Our org treats them as different records and each may have their own opportunities. However, in reality, if an opportunity closes for one of these accounts, I need the model to understand that the marketing touchpoints for the other ‘sister accounts’ must also be taken into consideration, there needs to be marketing attribution because the two are connected in real life.
- When the ‘fuzzy match’ is done, we use something called a support vector machine. These are machine learning algorithms that are normally used for classification, but we use it for regression here. This allows the model to understand complex relationships in the data with accuracy, not just linear relationships.
Running Marketing Scenarios
Now I could run some scenarios with the models. This is what I was really looking forward to doing: running the models as if there was zero marketing activity to see how the sales pipeline would have looked. Finally, I could isolate the effect of marketing.
This is something I am proud of in my career, so far! I have certainty that 10.3% of our net sales are directly attributable to the effect of marketing.
This also leads us to some interesting questions. Does marketing justify its existence? For example, if marketing generated €100 in sales but €100 was spent on the marketing activities, then the answer would be ‘no’. However, if we look at our organization, I calculated that for every €1 we spent on marketing, the organization gained €3.45 in profit (EBIT).
We can now visualize our whole funnel – from reach, engage activation, nurturing, through to revenue and the impact of each variable – all in one dashboard. The one you see below is built using Power BI:
This is only possible if you are ambitious and if you leverage the richness of Pardot (Account Engagement) data properly (most likely through the API).
Key Takeaways:
- Einstein Attribution rocks, allowing you to do things you couldn’t have imagined possible before – all by using a plug-in-and-play product.
- Leverage the Pardot API for data I/O. Once you understand the richness of Pardot data, you will be amazed at what’s possible.
- As a team, we’ve come so far – I feel like we have only just scratched the surface!