Follow each of the seven steps to build an email automated split test:
Create a Split Test
The first step in creating an automated split test is to establish the initial setup of the split test.
Navigate to Messages > Email New Email Split Test.
Enter a name for the split test.
If the split test is part of a campaign that is previously set up, uncheck Use this test name as the campaign name and select a specific campaign from the drop-down menu. There is the option of designating the split test as "Non-Campaign" as well in this drop-down menu.
💡 If the split test is not part of a campaign, keep this checkbox checked.
Click Apply.
Test Criteria
The next step is to select the criteria that is to be tested. Select one of the options from the dropdown menu to begin:
Delivery Date: Sending messages at a specific time on multiple dates.
Delivery Time: Sending messages on a specific date at varying times.
Subject Line: Sending the same message with multiple subject lines.
Message Content: Sending different email content altogether to test recipients.
From Name/Email Address: Sending the same message with a different from name or email address.
💡 A 10-way split is the maximum limit for testing. Our recommendation, however, is to adhere to an A/B or two-way split test.
Delivery Date
To test multiple groups of people across multiple days and see which day gets the better results, use Delivery Date.
Select Delivery Date from the Test Criteria drop-down menu.
Select the start time.
Select the number of messages to test.
For each test, give each group a name and specify the start date for each message to go out on.
Delivery Time
To test if a certain time frame performs better than another, use Delivery Time.
Select Delivery Time from the Test Criteria drop-down menu.
Select the start date.
Select the number of messages to test.
For each test, give each group a name and specify the start date for each message to go out on.
Subject Line
To test which subject line performs best, use Subject Line as the test criteria.
Select Subject Line from the Test Criteria drop-down menu.
Select the number of messages to test.
For each test, give each group a name and specify the start date for each message to go out on.
Message Content
To test how different content performs, use the Message Content split test.
Select Message Content from the Test Criteria drop-down menu.
Select the number of messages to test.
For each test, give each group a name and then add the content to each group.
📑 Learn how to add messages to a split test.
From Name/From Email Address
To test which "from name" or "from email address" produces better results, use the From Name/Email Address option.
Select From Name/Email Address from the Test Criteria drop-down menu.
Select the number of messages to test.
For each test, give each group a name and then add a from name and from email address to each group.
Add Messages
There are two options for adding messages to a split test, either Create New Message or Select A Saved Message.
⚠️ Repeat these steps if using Message Content as the test criteria.
Create a New Message:
Click Create New Message.
Fill out the message details.
Scroll to the bottom of the page and click Commit Message to add this message to the split test.
On the confirmation screen, click Commit Message To Split Test to confirm all message settings.
Click Apply to move to the next step.
Select a Saved Message
Click on Select a Saved Message.
Select the message to be used for the split test.
Scroll to the bottom of the page and click Commit Message to add this message to the split test.
On the confirmation screen, click Commit Message To Split Test to confirm all message settings.
Click Apply to move to the next step.
Audience Size
Choose to send the split test to the entire list or apply a filter to send it to a portion of the list.
To send to the entire list, simply click Apply to move onto the next step.
Test Types
Be aware the type of split test may influence the 'Winner Determined By' selection. For example, in a subject line split test, it would make sense to determine the winner by open rate, not read rate.
Begin by first selecting how the winner will be determined by selecting from one of the options from the drop-down menu:
Open Rate: Calculated by dividing the total number of messages sent by the number of contacts who opened the message.
Read Rate: Calculated by dividing the total number of messages sent by the number of contacts who read the message.
Click Rate: Calculated by dividing the total number of messages sent by the number of contacts who clicked a link in the message.
Click to Open Rate: Calculated by dividing the total number of messages open by the number of contacts who clicked a link in the message.
Conversion Rate: Calculated by dividing the number of conversions that were made by the number of contacts who clicked through the email to land on your website.
⚠️This option is only available if Listrak Conversion Analytics (Conversion Analytics if white-labeled) is turned on for the list.
Dollar per Thousand Messages: For every 1,000 messages, how much revenue was generated?
⚠️ This option is only available if Listrak Conversion Analytics (Conversion Analytics if white-labeled) is turned on for the list.
Next, choose either of the message performance options:
Winner Takes All: This option sends the messages to a sample size of the list to determine a winner, then after the time frame for testing is over, the remainder of the test subscribers will receive the winning message. ⚠️ This option is NOT available when conducting a Delivery Date or Delivery Time split test.
Click the radio button next to the name to select this option.
Adjust the slider to determine the sample size.
This option maintains the same ratio across every split group.
Click Apply to move to the next step.
Standard Test: This option sends the messages to the audience without a remainder group awarded.
Click the radio button next to the name to select this option.
Adjust the slider to determine the sample size.
This option DOES NOT maintain the same ratio across every group.
Click Apply to move to the next step.
Test Duration
Scheduling the send
Choose to start the test at a specific date and time, or begin the split test immediately. ⚠️These options are NOT available when conducting a Delivery Date or Delivery Time split test.
Determine the duration of the test
The test can run for hours or for days by choosing from the drop-down menu and using the plus or minus icons to increase or decrease the duration.
Winner Takes All: After the test duration is over, the winning message will be sent to the remainder of the group.
Standard: The winner will simply be calculated and reported on after the test duration is over.
Optional Multipass
Selecting 'On' will send a MultiPass of the winning message to non-openers after the test duration has ended.
Winner Takes All test type options:
Resend the winning version to all TEST groups after X hours once the split test has completed.
Resend the winning version to all REMAINDER groups after X hours once the split test has completed. ⚠️These hours need to be greater than the test group.
Projected Send Date will calculate the date and time BOTH groups will most likely receive the follow-up message.
Standard test type options:
Resend the winning version to all TEST groups after X hours once the split test has completed.
Projected Send Date will calculate the date and time all TEST groups will most likely receive the follow-up message.
Click Apply to move to the last step.
Confirm and Publish
Review each setting before starting the split test. The option is given to Edit each section except for selecting a different Test Criteria.
To begin the test, click Start Test.
💡 The option to view or make edits to the test is available up until the time the test is scheduled to begin.
