After you have run an email split test you may want to view the results of these tests to see how contacts engaged with your emails. There are multiple ways to view the reporting of your split test, depending on the goals of your reporting.
Message Activity Report
The Message Activity Report allows you to view the reporting of all broadcast messages sent during a given timeframe.
If using the same campaign name for a split test message and additional messages, you will be able to see the performance of all messages in a specific campaign. An A/B icon will appear before the subject line of your split test message. This is a great way to test different variables in your messages, while still keeping an eye on higher-level metrics, such as overall campaign performance.
Split Test Activity Report
The Split Test Activity Report is available in the navigation under Analytics > Split Test Activity.
💡 Use the date range picker to select a timeframe and see the performance of any split tests sent during the selected range.
The data available in the Split Test Activity Report is the same as the data available in the Message Activity Report, but limited to the split tests sent on a specific list. In this report you can compare the metrics of the different variations of the test.
The group name that you specified during the split test setup is available in the reporting. In icon (trophy) will allow you to easily identify the winning version based on the metric you selected during setup. The other variations are identified by the envelop icon and, the pie chart icon indicates the remainder who were sent the winning version (if a winner takes all split test type was chosen).
This report is a great way to view the metrics specific to split test messages and provides a quick indicator of the winner and additional messages to make it easy to put the results in context.
Automated Testing Results
The automated testing results can be viewed in the navigation under Authoring > Automated Testing > View Test Results. This report is useful both for viewing the results of split tests have been completed as well as keeping an eye on that status of split tests that are scheduled or running.
Currently scheduled or running split tests will appear in the Your Test tab. You can edit a scheduled split test from this screen.
All split tests that have been conducted in an account will appear on this screen. Use the search function to view split tests sent on specific dates, use the filter test option to search for tests based on test criteria or winning metric, and by test name. Click update to search for the tests that meet the criteria.
Hover over a specific test to see high level information, including: test criteria, winning metric, type of test, and number of variations. Use the Results button to dive deeper into results.
On the results screen, you will a breakdown of each of the different possible winning metrics for each message. The first chart available, will be the winning metric.
⚠️ A warning symbol next to one of the metrics indicates that the non-winning version of the split test performed better based on the specific metric.
Hovering over an individual bar in the bar chart will display the metrics for the test and current results. The test result includes the metric that was calculated at the end of the split test duration. The current result also includes metrics that have been collected after the test ended. Viewing these metrics can be helpful to identify if behavior is different for short-term engagement vs. longer-term engagement.
Tracking all metrics can also help you to identify if different behaviors change based on different versions of a message. For example, you may find a certain type of subject line leads to increased open rate, but the other type leads to increased conversions.
Clicking the Test Progression button opens another report of the different winning metrics available. In this section you can see how the different versions of a message performed based on a specific metric over the course of the test duration. These charts can be another usefult tool to see if behavior changes over time based on the message sent.