Last updated - July 8, 2021
A/b Tests for Ecommerce? YES! For Hubs? Maybe not so much …
Before we dive into the philosophy, and a few techniques, we should probably cover exactly what an A/B Test is. Once you get the basic concept, the value will be obvious, and you will probably think of many ways it can be applied.
An A/B Test is exactly like what it sounds like, and is named specifically. You are testing A versus B and vice versa. This method is a scientific approach, which is why they use the word ‘test’. So, in a basic A/B test, you have:
- A desired goal (sale, subscription, subsequent pageview, click, higher CTR, Higher CONV RATE, etc …)
- A portal to that goal (generally a web page that leads to, or facilitates the desired goal). Portal / webpage can be a product page, an article, a landing page, and so on.
- Two different ideas, or approaches for that portal (which from here on out we will call a Landing Page, for simplicity).
- The first idea you want to try will be called: A
- The second will be called: B
- A method to determine significant success (measured in ‘difference’).
- A method to keep track of the data, measure it, and produce actionable information.
- And a method to implement the change(s) inspired by the results of the test.
So, in simpler terms: An A/B Test is a procedure that measures the efficacy of two different approaches to the same goal, with a focus on determining which is better and why.
Just to be clear … Can I get an example of an A/B Test?
I think that’s a good idea. Let’s show you a quick example of an A/B Test, so you can put the theory into motion in your mind.
With the primary picture (graphic [top]) for this article in mind, let’s say you have a store that sells Mountain Bikes. You’re using a variety of methods to drive traffic to your site that sells mountain bikes. So, for this particular test, you want to increase the number of sales for a specific type of bike: The Panther Mountain Bike
The way your store is built, someone must first look at the product page before they can make a purchase (which makes it a required step in our test). Once on the product page they can either: A.) leave your store, B.) visit another page on your site, or C.) purchase the bike. Our goal for this test is to increase the number of people that purchase a bike directly after viewing this page.
The A/B Test would go like this
- You would create two product pages for the same exact product.
- Each page would be different in some way.
- Let’s say in this example you change the headline, the product picture, and you move the “buy now” button to a different place on the page. But, you keep the sales copy exactly the same.
- The key performance indicator (KPI) in this example is: number of panther bikes purchased
- Have a way to measure and record the activity of the test. This is critically important, and we’ll discuss this in more detail below.
- Launch the test and collect the data.
- Once the data is collected, and a statistically significant threshold has been achieved, apply the winner as a permanent change to your site.
What do you mean, ‘statistically significant threshold ‘?
This concept is of particular importance. You need a certain amount of data before you can deem your results “reliable” and “actionable”. Let’s look at two extreme examples:
- After 10 sales, you decide to end and measure the test results: Page A produced 5.45% higher sales than Page B.
- After 1000 sales, Page B produce 3.96% higher sales than Page A.
In our above example, after 10 sales, it looked like Page A was the winner, but because your sample was so low (only ten sales) you were not looking at statistically reliable data. If you were to implement the results after only ten sales, in this case, you would have picked the worse of the two iterations. After 1000 sales, even though the increase is lower (only 3.96%) it is much more statistically reliable, and should be implemented on your site as a permanent change (or until another test performs better!).
Why should I A/B Test – What’s the point?
The goal of an A/B Test is to identify, scientifically, a more successful approach to your Ecommerce goal. Our web pages, and stores, especially the smaller ones, are put together by web designers, graphic designers, copy writers, and sometimes just regular business persons with absolutely no training in any of the above disciplines.
So, while the one working on the site might think the navigation architecture is brilliant, the product image is stunningly artistic, or that the sales copy was written with such grace that it could make a hardened construction worker question his place in reality, none of that matters if the site doesn’t convert visitors into sales. And I’ll tell you this: Often times what you think will work doesn’t, and what you wouldn’t expect to work, does.
That is the primary reason we test. It takes the opinion out of the equation and measures raw performance. So if you’re serious about your Ecommerce store, and you’re not testing it CONSTANTLY, then you are missing out, big time.
Well, I understand what an A/B Test is, and the theory, but how the heck do you actually perform one? Is there software to do this?
Excellent question. Yes, there are many ways to do this.
One of the most popular, and powerful, ways to do this is to advertise with Google’s program called adWords, which if you’re reading this article, you should at least be vaguely familiar with if you’re not already sleeping together.
There are other software programs out there designed to manage A/B Testing on your site, and they are usually part of an analytics suite. But, for the sake of simplicity, and freeness, if you’re aready using adWords to drive traffic to your site, use their A/B Testing.
A/B Testing is just one of several tests you can perform
We’re talking about A/B Testing in this article, because it is the simplest and quickest way to get actionable results; Which is better? A or B? Simple. But there are other types, types that get incredible complicated and require throngs of traffic.
If you research this topic further you will inevitably run into the Multivariate Test. The Multivariate Test is also very powerful, maybe in some ways more, but it is not always appropriate. As its name suggests, it is a test that deals with multiple variables. So, lets say you want to test three different headlines, three product pictures, two ‘buy now’ buttons, and two paragraphs of sales copy. Each variation is a variable in the Multivariate test. The test will create every possible combination of the above variables and deliver it to the customers at random. After a significant amount of data is collected and analyzed, the best performing combination can be identified. This method is very intense, and should only be conducted by professionals, on a site with lots of traffic.
As always, just let me know if you have any questions, or would like me to clarify something for you.
