What is A/B Testing and how does it work?

Last updated on March 14th, 2022 at 01:34 am

In a perfect, more static world, an entrepreneur would create a product, set up a platform to sell it, and go along their merry way, making sales and enjoying profits.

Today’s Challenges

For better or worse, this is not a static world, and frequently, business owners need to make updates to their squeeze and sales pages to increase business flow and re-generate excitement about the site and the product or service.

But how does the savvy business owner go about this?  Is it best to just tear everything down and rebuild?  Not necessarily!  Sometimes, potential customers just need a little enhancement to encourage them to click here and complete a sale.

The tricky part is figuring out what little tweaks or changes might be more enticing to an audience, and that’s where A/B testing comes in.

In the marketing world, there are really no such things as “best practices.”  What sells two million ebooks on one website might not move a single page on another site. If there’s anything marketing has taught us, it’s that audiences are picky, and motivation is impossible to predict. 

What Is A/B Testing?

A/B testing, also known as Split Testing, is a method that allows marketers to create two or more versions of the same page and run them simultaneously before audiences, to measure which version has a better conversion rate. Its goal is to improve metrics such as clicks, form submissions, downloads, or purchases.

A split test is a safe, low-cost way to determine how small changes to your pages can impact customer behavior.

The basic principle of A/B Testing is this:  come up with one small change to a website page or an email.  Grab two equal and random test groups.  Show version A to one group, and show version B to the other group, and analyze overall performance over a period of time.  Whichever version generates the best results is the winner.

Now, before you blast off with a hundred different versions of your website, hold on.  A/B testing is a controlled experiment, and there are some things you should know before you start testing on a whim.

Read also: How to Find Your Superconsumers

The A/B Testing Process

First, figure out what is to be tested.  There are so many things that customers see.  You may want to determine if the location, color, or size of a feature such as a call-to-action button will make a difference. You may want to change layout, or the design of a page.  Maybe it’s the wording that could use some revision, maybe it’s the overall customer experience.

Whatever items might be up for renovation, choose one and only one.  If you make multiple changes at once, you’ll never truly know if it was the size of the button that made people click on it, or the rewording of the product details that created the conversion.

Stick to one thing, and one thing only. 

Next, create a hypothesis of sorts.  What are you measuring, and how are you measuring it.

A hypothesis might be something such as:

“If I change the layout on the product page, I will see more conversions and purchases.”

While you’re creating this hypothesis, think of how you plan to measure success.

If you have two more sales during the time frame of the split test, that might just be coincidence.  If you have 80% more clicks on a call-to-action button, that would actually demonstrate some movement. 

The parameters of the A/B test must be set before things get rolling.  You’ll need at least two sample sets, the A and the B.  A is the control, meaning this group will see everything as it is now. 

Set B will see the new and hopefully improved option.  These sample sets will need to be of equal size, and chosen randomly from your audience group.  This is not the time to try to target marketing, because that could skew the results!

Once the sample sets are ready to go, determine how long the test will last.  It is important that the A/B test receives enough views, and gives the audience enough time to make decisions.

If conversion typically requires more than a week, don’t cut the split test off at a week just to hurry the process along.  It is crucial to test both options simultaneously, because timing can skew everything.

As we’ve discussed, target audiences have a target timeframe, and this is not the time to target.  This experiment relies on equal playing field with a single changed element!

When the Test Is Over

It’s not a bad idea to ask for feedback from real life users once the test has concluded.  Feedback is always helpful for business owners, especially in the tricky and delicate world that marketing can be.

If customers are willing to explain why they clicked on a certain feature, this information can be invaluable for setting up future features. To do this, you can send a survey to your list using SurveyMonkey.

Many times, the results of an A/B test provide an obvious change in behavior, meaning that a change has very clearly made a difference.  Sometimes, however, there’s not a clear difference or change.

In this case, it’s back to the drawing board.  Perhaps that particular feature did not require refreshing, but a little remodeling in another location can do the trick.  Remember that websites are not static.  Things can – and should- change frequently.

A/B Testing Tools

For those who plan to conduct A/B testing, make sure you have an A/B testing tool on hand.  This tool will take out the guesswork and possible confirmational bias a business owner might have towards a particular result. 

There are several testing tools available on the market right now, so do a quick search and determine what capabilities, functions, and overall results you’ll want to test and review to choose the best tool for your site. 

If you are using WordPress as platform for your business, many of the WordPress page builders include the split testing feature. I suggest you to have a look at this page for more information.

In the image below, you can see an example of A/B Testing Statistics using Instapage (click to check their website).

Some of the most common third-party softwares for building landing pages featuring A/B testing tools, are Clickfunnels, LeadPages, and Instapage. All of them offer and a native split testing campaign engine, and you can integrate their landing pages into your WordPress installations.

The things that build a successful website can sometimes seem like a mystery, but processes such as A/B testing allow business owners the opportunity to try small changes before undergoing a huge overhaul of their website and sales pages.

Creating a control and a sample change can also help gain feedback on what options (such as size, placement, color, layout, and wording) work for a greater percentage of an audience.

Read also: Meaning and Differences between UX and UI Design

The overall goal is to make a website as attractive and functional as possible.  Split testing is a great way to allow business owners to gauge how customers use a site to make it ever more attractive and functional, without spending loads of time and money on changes.