Your automated campaigns provide a great touch point for your contacts. They also provide an opportunity to learn more about what motivates your contacts by split testing your campaigns. From message cadence, to subject lines, to message content, or including an offer you can test almost any variable in your campaign.
⚠️ This guide sets up a traditional A/B split test which tests one variable between two different groups, such as message timing. You can create as many different groups as necessary for your testing, but should only test one variable at a time to easily identify which is most successful.
Read more below on how to create an A/B split test in Listrak Conductor. Need some inspiration before you get started? Check out our example!
Setting Up Your Assets
To begin setting up your split test you will need a few assets on hand. This guide will review the setup of the last component.
Your Split Test Messages
Coupon Pool, if using
Create the Split Test Segments
When creating a split test in Conductor, a contact will be assigned a segment to indicate the version they should receive. Prior to setting up the logic in Conductor, you need to set up a profile field to store the value of this segment.
On the list you are building your Conductor Conversation, navigate to Contacts > Email Contacts Profile Fields.
Click on the filed group associated with your conversation.
If you do not have a field group, click Create Field Group.
Name your Field Group.
In the new field input, call the field Split Test.
Select Numeric for the data type.
Click the Floppy Disk Icon to Save.
Creating Your Threads in Conductor
When creating a split test, you may already have an existing thread with the experience your contacts are currently receiving. In this case, you will need to create two additional treads 1) the test logic thread (in orange) and 2) the B thread (in purple). The B Thread contains the new experience with the changes to the variable you are testing, such as messaging timing. If you are creating a new Conductor conversation and including a split test, you will also need to create thread A (in blue).
The steps below add a split test to a currently active conversation.
Create Thread B
Thread B is the new experience contacts may receive when progressing through the split test. Thread A and B should be the same, except for the specific variable you are testing. You may be able to see this difference in your Conductor interface, for example if the wait times are different or the message subject line, but some changes may not be visible, such as message content.
Click on the green plus sign below any existing threads to create a new thread.
Use the arrows on the left side to move this thread below Thread A, but above any other threads.
Click on the Trigger Step and in the Tigger Step Name field, adjust the name to match your goal. E.g. Shortened Time Thread.
In your events panel, click the No Events Required Checkbox.
📌 No events are required because contacts will be sorted by your test logic thread set up below and then sent to Thread B.
Click Update.
Click in the green plus sign between the Trigger and Terminate Steps and add them to match Thread A, or your goal, if you are not mimicking Thread A.
Configure each step, only changing the properties to match the variable you are testing.
Update Thread A
Now, you need to updated the properties of the experience your contacts are currently receiving. When the conversation is published with the split test, contacts should only receive Thread A after they have been sorted in the logic thread you will set next. So, you will need to stop a contact from being eligible to enter the thread when taking an action on your site.
Click on the Trigger Step for Thread A.
In your events panel, click the No Events Required Checkbox.
Click Update.
Creating a Test Logic Thread
The test logic thread will sort contacts and send them to the different experiences you are testing in your split test. In the action step you will add information to the profile field you created earlier. The field will be populated with a numeric value corresponding to their experience (in this example, a value of 1 will correspond to Thread A and a value of 2 will correspond to Thread B). You will use the Go To Step Conductor step type to move contacts to their new experience.
Click on the green plus sign below any existing threads to create a new thread.
Use the arrows on the left side to move this thread to the top.
Click on the Trigger Step to adjust the properties.
In the Trigger Step Name field, adjust the name to match your goal. E.g. Split Test Timing Logic.
In your events panel, click the plus sign next to the events associated with your campaign.
💡 The events should match the event(s) now greyed out in Thread A.
Click Update.
Next, click the plus sign after the trigger step and add a new Action Step.
Set the following properties in the Action Step.
Select the Create Split Segmentation from the Action Type drop-down.
Next, select the profile field created earlier to save the split value to in a contact's profile.
Last, add a number of segments. This will correspond to how many versions you are testing. A traditional A/B split test will contain 2 versions.
If desired, you can override the default name to easily identify what is occurring.
Next, you will create two go to steps. These will send contacts to the new location in your Conductor Conversation based on the value set in their profile by the action step.
Click on the green plus sign after the action step and add a Go To Step.
Click the green plus sign after the new step and add another Go To Step.
The test thread has now been created. You'll set up the properties for the Go To Step next.
Setting The Logic of the Split Test
Once your threads have been created, you'll set the properties of your go to steps to direct contacts to either Thread A or Thread B of your split test. The first go to step will direct contacts to thread A and the second will direct them to Thread B
Click the first empty Go To Step in the test logic thread to adjust the properties.
Rename this step "Thread A."
In the goto tab, select the first Wait Step in Thread A as your destination.
In the Segment Filter tab, Click Edit Filter.
Select Profile Field Groups.
Then the name of the filter group that matches your campaign.
The field will be the split test field you created earlier
And the value will be equals 1.
Click Update when all settings are complete
Click the second empty Go To Step in the test logic thread to adjust the properties.
Rename this step "Thread B."
In the Goto tab, select the first Wait Step in Thread B as your destination.
You will not need to adjust the segment filter settings for this go to step. Not adding a filter means all contacts will be directed down to Thread B, however you have already removed all contacts eligible for Thread A, so the only contacts who are in the go to step are those eligible for Thread B.
Click Update when all settings are complete.
Publish Your Split Test
Publishing your split test allows contacts to start receiving the campaign with the new test variable.
Click Publish to save your changes and make your conversation live.
Example Split Test
In the split test below, a Shopping Cart Abandonment Campaign is being tested to determine if there is an impact when changing messaging cadence. Contacts are only able to enter the Logic Thread, all other threads are marked as No Events Required and contacts are directed there via go to steps. When a contact enters the thread the Action Step will write a value of 1 or 2 into a profile field. Those contacts who have a value of 1 in their field are directed to thread A and all other contacts are directed to thread B.
Threads A and B have identical message steps and action steps to set a coupon, as well as go to steps to exit contacts out who do not have cart information to populate the email. The only difference between the two threads is the wait time between messages, which is the variable being tested by the split test.