A/B Test Broadcast Emails

You can convert a broadcast email to a split-test that can test up to 5 email variants. 

Note: that currently, split-testing is only available in the Send a Broadcast section.


  1. Click the A/B Settings button when you are designing your broadcast email

  1. Click the Enable A/B Testing button

  1. Name your split test

  1. Add variants

You can add up to 5 email variants to split-test. Then, you can move between variants by clicking on the variation buttons. When you add additional Variation it copies the previous Variation Subject, preview text, and design.

Here are the most common elements that are split-tested:

  • Call to action
  • Subject line
  • Preview text
  • Overall design
  • The overall message: Headline, Body and Closing text

Pro-Tip: Click on above image to view larger

Note that you can quickly delete a variant by clicking the trash can icon.

Also, note that you can cancel this split-test altogether by clicking the Cancel this A/B Test button at the bottom.

  1. Choose the recipients for the email.
    • Use a saved search you've created before (e.g., "Newsletter List")
    • Add recipients based on a tag applied
    • Or click "New Search" to search directly within the Email Builder.

Pro-Tip: Use New Search if you want to send to contacts that contain a tag but also excludes a tag.

  1. The slider allows you to control how emails are distributed



For example, in the image below, 10% percent of the emails will be split evenly between the email variants that you create. Then, based on the winning metric (which is explained in the next step) the system will distribute the rest of emails (90%). If you prefer, you can split test your entire distribution list by moving the slider all the way to the right.

  1. Choose the winning metric for this A/B test
    • Click Rate: Also called, "click-through rate" this is the percentage of recipients that clicked a link in your email. This is a very common way to measure the effectiveness of your email's call to action.
    • Open Rate: Another way to measure the success of an email is the percentage of recipients that opened your email. This is also a common metric for measuring email effectiveness, but not as scientific as click rate due to the false positives and false negatives that is inherent with email clients.

  1. Determine the time period that you would like to run the test

You can choose to run your split-test from 4 hours, all the way up to 7 days. The time period you choose will primarily depend on your list size and how active your list is. A split-test to a large, active list may only take 4-8 hours. A small or less active list may take longer to get an accurate picture of which email variant was the most successful.

  1. When you are ready to run your split test, click the Review and Send button at the top right

    • You will be able to see details on the variant distributions prior to starting the test.

A/B Testing currently doesn't have the ability to schedule to send the email on a specific date/time
  1. Click the Run A/B Test button to start the test

Pro Tip! The Broadcast Report pages will help you can better understand your results of your A/B Test.


 

Was this article helpful?
Thank you for your feedback!