How to Design an A/B Test

How to Design an A/B Test

In last week’s post, I talked about what A/B tests are. This week, I am taking it one step further and breaking down the steps required to design and set up an A/B test.

1. Pick a tool

There are a number of tools available to help you with A/B testing:

  • Google Website Optimizer
  • Visual Website Optimizer
  • Optimizely

I have used and can recommend Visual Website Optimizer – it’s got some neat conversion optimisation features and is RIDICULOUSLY easy to use.

2. Pick a page

The next step is to figure out where you are testing. The first port of call is usually the homepage since it gets the most visitors. Other pages that you will want to consider are:

  • Product pages
  • Category pages
  • Shopping cart pages
  • Pages with high page views
  • Pages with high bounce rates from entrances (ie visits where the page is the landing page)
  • Pages with low conversion rates
  • Pages for high margin products/services

As a rule of thumb, you want to get the basics of your main pages right before you look at fine tuning the other pages. Of course, there are exceptions to every rule – if are a game retailer and you have a blockbuster coming out soon, then you want to A/B test that page, stat!

3. Decide which elements to test and design variations of them

You can literally A/B test anything on a page – however, you want to be careful that you don’t bite off too much as the statistics get harder to interpret as you add more elements. Try testing one or two elements with two variations (control and an experiment) to start off with. Some questions you can ask to decide what elements to test are:

  • Is there a headline on the page?
  • Are the calls to action visible? Are they competing with each other?
  • Are there hero shots on the page?
  • Are there any areas of clutter on the page?
  • Is there any copy on the page?
  • Are there any images or logos on the page?

The astute reader will notice that there are a lot of dumb questions on that list. Yes, every page should have headlines, hero shots, content, calls to action, etc – so OF COURSE you can test different variations of them!

Also, using click maps or predictive heat maps (Visual Website Optimizer includes this as a standard feature) are a great way to find out where people’s focus is and where they are/aren’t clicking.

4. Replace the elements to be tested

Using the tool of choice, design your variation(s). Visual Website Optimizer has a simple What You See Is What You Get (WYSIWIG) editor, which makes the job a LOT easier. Sure, it’s a little buggy and far from perfect…but remember that A/B Testing also means Always Be Testing. So ditch the hunt for perfection and just plunge in there!

5. Add the Javascript code to the website header

The last step is to add the Javascript code provided by the tool to the header section of the page you are testing. This is essentially a 2 minute tweak for your web developer.

Running the Test

Now that the test is all designed and set up, it’s time to run that puppy. I’ll go into more detail on how to run the test in my next post. As always, stay tuned!

E-Web Marketing
Follow us
E-Web Marketing

We’ve been in the digital marketing field for over 18 years and worked with hundreds of Australian (and international) businesses to grow their web presence. Specialising in SEO, search ads (PPC), social media, content marketing, email marketing and conversion rate optimisation.

Get your 100% free SEO Analysis


    100% Free. No obligation. Done by a human.