AB(CD)-testing in email

To create a successful AB, ABC or ABCD-test, first you should make sure that you have recipient lists that are big enough to generate a statistically adequate result.
If your have recipients with more than 10.000 per list it will be worth while. But if you have a lower amount of recipients you will, most probably, not be helped by an AB-test. Simply because the test groups selected will never be big enough to give a winning result that you can rely on.
PLEASE NOTE: There is not possible to have a send out speed limit when you do an AB(CD)-test.

How to set up a AB-test in Carma?

1. Decide the type of AB, ABC or ABCD-test you want to do.

2. Choose what part you want to test with your different groups. With Carma ABCD-test you can choose to test on Subject Line or Email (Content)

3. Adjust the size of your different test groups. To the right you can see how many procent of the recipient that will receive the winning version.
The winning version is the A or B (or C or D) that generated the best result in opening rate in this example.

PLEASE NOTE: Remember that the winning version is based on the result at the time you have set. So the result may actually change after the winning version has been sent. Which may indicate that you should set a longer evaluation period next time.

Statistics below are showing that the B-split was the most successful version at the time for when then winning version was sent out.
But, at the same time, the A-split was actually more successful over time.

Create AB(CD)-test on content

When you choose to create an AB(CD)-test based on different content you will get the possibility to create generic content that should be the same for all your different AB(CD)-tests. You can also choose to make adjustments for each version (A,B,C and D).

1. You start with the Shared Content, just hit the “Content” button and start to edit:

2. Then you can proceed with A and/or B:

Content A:

Content B:

You can copy specific slice content, start by clicking on this icon in the block you want to copy, then choose where you want to copy it:

For more information about Best Practice on AB(CD)-test – Please click here!