Account Engagement (Pardot) A/B Vs. Multivariate Testing (+ AI)

A/B testing enables you to test one change between two versions of your marketing assets. In the context of Account Engagement (Pardot), A/B testing is for list emails*. The benefits are that it’s quick to set up and can work even with a small sample size (of course, the greater the sample size, the more significant your results will be). 

On the other hand, multivariate testing compares multiple changes between versions of your marketing assets. In the context of Account Engagement (Pardot), A/B testing is for landing pages. Multivariate tests take a little longer to set up and require a larger sample size.

*Note: A/B testing is available for emails sent via the classic email builder. 

This guide will walk you through how to decide between the two types of tests, how to set up A/B testing, and how to use AI innovations (i.e ChatGPT) to get up and running faster. 

Account Engagement (Pardot) Test Preparation

Experimenting with marketing assets requires you to do preparation to ensure that you can track what you’re testing, and draw sensible conclusions from the results. 

You can use the pyramid below, a hierarchy to prioritize testing factors that can deliver the greatest results:  

• Value proposition: “What’s in it for me?” (where the “me” is your buyer). 
• Layout: Are you highlighting the right points, or is your layout busy with competing messaging? 
• Credibility: Are you building the buyer’s confidence, for example, thought leadership, expert quotes. 
• Social proof: Do you provide answers for when the buyer asks: “What did people similar to me choose, and were they happy with that decision?”

Next, follow these five steps:

1. Determine your goal. Is it to increase clicks? Improve landing page conversion rates? 
2. Create a hypothesis. Select a factor from the hierarchy above. If you make a change, what would you guess happens? 
3. Create alternatives: Change something about your current assets (copy and implement those on the copied asset).  
4. Confirm your measurement: Ensure your tools are set up properly to record the metric you’ve chosen. 
5. Execute your test: Test alternatives with live traffic, using the right method (A/B test or multivariate)

Source: Marketing Automation Unleashed

Account Engagement (Pardot) A/B vs Multivariate Testing

Set Up Account Engagement (Pardot) A/B Testing

With Account Engagement (Pardot) A/B testing, you can test the following factors (remember that it’s best to test one at a time!):

• Email body content
• Subject line
• ‘From’ (sender) name
• Reply ‘to’ address

1. Navigate to Account Engagement Email → A/B Tests.

2. Create a new list email. Select the Enable A/B Testing checkbox. 

3. Select an email template. A copy of your existing content will automatically be created for version B.

4. On the building tab, you will be able to switch between versions A and B.

5. On the sending tab, you will be able to change the “from” (sender) name, “reply-to address”, and/or subject line between versions A and B.

6. Continue scrolling down, and you’ll find the section to configure how your A/B test should be run. Set how long you want the test to run with the sample size before a winner is selected (any duration from 1 hour to 30 days), the criteria you want the winner selection to be based on (opens or clicks), and the % of your send list you’d like to use for testing. 

7. Once the email is sent, Account Engagement (Pardot) will send the email to the sample group, then record the results based on your selected criteria. When the experiment time frame is up, the winning version will be sent to the remainder of the send list. 

By following this hack, you can also perform A/B testing with Engagement Studio:

Leveraging ChatGPT in A/B Testing

Marketers have already been tapping into the power of GPT to write email and landing page copy, marketing plans, and advice for data breaches. 

Einstein GPT brings the power of GPT into the Salesforce interface. We saw a marketing use case during the release, where a marketer wants to create a landing page for an upcoming event, asks Einstein GPT to create a landing page template and campaign messaging – which they then ask to be made shorter.

While this is impressive, there are ways you can use ChatGPT on the fly to get the tedious task of amending copy done even faster.

1. Head to ChatGPT, and set up an account. 
2. You will arrive on the screen with suggestions, and a text box below. Ask ChatGPT to make your copy more formal/exciting/funny… whatever you choose! 

I asked Chat GPT to generate different versions of one of our SFB Course blurbs. The first is a formal version, the second exciting, and the third funny. This is a quick way to generate your content to meet different tones.

Summary 

This guide explained how to decide between the two types of tests (A/B testing and multivariate testing) and what’s required for each. You should now know how to prepare a plan for testing (so that you’re not just testing blindly). 

Setting up an A/B test in Account Engagement (Pardot) is actually straightforward once you know what you want to test. Finally, take the legwork out of creating variations in copy with AI innovations, i.e ChatGPT.