A/B testing is one of the standard ways to test two different products against each other. Savvy ecommerce business owners use this regularly to maximize their profits. It is also used to test landing pages for those who run ad campaigns. This kind of testing is not just for big business. You too can run A/B testing to optimize your website design to increase the number of visitors that sign up for a newsletter, subscribe to a feed, purchase a product (e.g. a domain) or any other action. Here is how to go about it.
What is webpage A/B testing?
A/B testing for websites is where two (or more) versions of a webpage are compared to each other to see which best encourages the visitor to take a particular action. Each visitor is shown a particular version of the webpage at random and then any conversion is recorded. As the number of visitors increase, a pattern starts to emerge in favor of one version or another and when this reaches a statistically significant difference, then the experiment is over. Don’t worry. You don’t have to do statistics yourself.
Using Google Website Optimizer
I use the Google website optimizer so I don’t have to do the statistics myself any more. This tool allows you to do experiment from the simple to the very advanced. I strongly suggest you stick to simple whole page experiments to start. Not only are these easier to run but they also are better for websites with lower traffic.
Traffic and website statistics
To generate a result, a statistically significant difference needs to accumulate. This depends on the traffic you get as well as the complexity of the experiment you are running. For example, if you are comparing two different webpages you will need a lot less visitors compared to one where you are comparing ten different versions. Put simply, you are splitting your visitors across two pages in the first example whereas you are splitting them across ten different pages in the second. Therefore you need a lot more traffic to get enough visitors on any one page in the second example.
Using the website optimizer
If you are using Google Analytics, then you should be easily able to set up an experiment. There is a lot of help on the tool and how to set up an experiment. Ignore instructions on multivariate testing unless you are feeling very adventurous and have a lot of traffic. Create a second version of a page already on your website and upload it to the server. Try running a simple A/B experiment so you understand how it works.
It is a very powerful technique that doesn’t depend on how you or your website developer feels about design or layout. It gives you objective real time data that you can use to optimize your website.
The Google blog had a good introduction and example of A/B testing
{ 2 comments… read them below or add one }
Good stuff. It would be interesting to see some stats and different page layouts/techniques that you had success with in the future.
Well there are two reasons I didn’t give examples.
The first is that this information can be used by competitors so it would not be smart to share this data for my websites
The second is that this information would either be useless or misleading to most people (other than savvy competitors) and I don’t want to misinform anyone. The data is very specific to the test case you are looking at and really cannot be generalized for the most part. So instead I am encouraging each person to set up their own test cases so they will have accurate data on their own websites.
The second link shows an example from Google if you need a generic example.