A/B Testing meaning

A/B or split testing is an optimisation technique. It involves presenting different versions of a marketing message to potential customers and tracking the performance of each message in terms of conversions.

To perform an A/B split test properly you need to plan it and measure the outcomes carefully. You also need to make sure you remove any external bias to the results. For an extreme example it is no use testing a message that offers a special weekend discount if you send half the emails out on Friday and half on the following Monday morning after the discount period is over. The response rate is bound to be different.

There are other factors that can bias results: time of day, location of reader, day of week and so on. To deal with this situation it usually makes sense to alternate the offer you are presenting to sales prospects so that half see one version and half see the other in strict rotation.

One technical issue that needs to be addressed is to make sure that someone seeing offer A, sees the same offer A if they return to your website. Likewise, someone seeing offer B needs to be shown offer B if they return. This degree of control over testing can usually be done with cookies.

Split testing can be applied to email marketing too. For example, the autoresponder Aweber allows you to split test email broadcasts. When you send out a broadcast as part of an email marketing program, you can write up to 4 different versions of your emails. Aweber will then track the conversion rates of each version.

An effective way of using the results of an A/B split test is to run a test on a subset of your email list.

Suppose you have 50,000 names on your list. You can do a split test on 5,000 of them (you need a good number to generate statistically valid results). Once you have determined which email message generates the greatest conversion rate you then use that version to email the remaining 40,000. This will maximise your profitability.

Split Testing Tools

Some powerful split testing tools are now available to help you do split testing. For example, take a look at the Visual Website Optimizer.

Google Optimize

Google used to provide a good optimisation tool called Google Optimize but they decided to withdraw the service. Here’s their announcement “Google Optimize and Optimize 360 will no longer be available after September 30, 2023. Your experiments and personalizations can continue to run until that date. Any experiments and personalizations still active on that date will end.”

Google is creating problems for itself by sunsetting various services they have launched. As this happens more and more, they will find that developers and other mainstream web users will avoid using any future services they launch precisely because they suspect that Google may sunset new services at some indeterminate time in the future. This means that it’s not worth relying on Google anymore. Recently for example, they have sunsetted the Google Domain name services