Split Testing 101: Simple

Hacks for Major Growth

Copyright © Splitly 2020 Splitly.com

WHAT IS SPLIT TESTING?

Split Testing is running controlled experiments to improve your website performance. A split test will tell
you, with the data to prove it, if a particular change has had an impact on your site or Amazon
listing. This can be in the form of improving clicks, conversions, sales, sign ups, or anything else on
your site.
In running split tests, website traffic is randomly directed to different versions of a web page, and user
activity is measured to see which version converts the best for the desired metric.

When running these experiments, there is an original version (known as the control), and a variation of
the original that aims to improve on the control version. After running the experiment for enough time to
generate a meaningful set of data, known as a statistically significant data set, testers can verify which
version produces the greatest positive impact based on the results.
WHY IS SPLIT TESTING IMPORTANT, ESPECIALLY FOR
AMAZON SELLERS

Split testing is one of the most important ways to improve your conversions and grow your sales. This is
especially true with Amazon, where you are limited by what changes you can make to your listing.

A regular website can change an infinite number of things, from text, to headlines, colors, buttons, user
flow, and much more. However, Amazon forces you to work within its strict parameters. This means
that every character of text, or image, is extremely important. Without testing anything, you are op-
erating on your assumptions of what will work, or worse yet, you are basing your listing on what your
competitors are doing. This can be disastrous.

Wouldn’t it be more effective if you listened to what your customers need? By running split tests, you
are creating a way to have your customers tell you what works and what doesn’t, by measuring their
behavior.
HOW TO PLAN YOUR SPLIT TEST PROPERLY

Before you start testing anything, you need to figure out what are the main metrics you want to
im- prove. Is it getting more visitors to your page (by increasing the number of clicks to your
page), is it getting more sales (by tweaking your copy), or maybe improve your ranking for a
given keyword (by adjusting your headline).

Think about all of the metrics that are most important to you, and what factors affect that

metric. From there, you can start crafting your tests!

HOW TO MEASURE YOUR SPLIT TESTS PROPERLY

To get test results that you can feel comfortable about, you will want to run tests with 90% or 95% sta-
tistical confidence. As far as time frame, you will want to run your test for at least one week per variant,
which means two weeks for a full split test. This will help you mitigate some of the variation that a
day or time of the week can cause.

If you run your tests to greater statistical significance, you reduce the risk that you are making any
changes on misguided data.

How do you figure out your statistical significance? Tools like Splitly will do this automatically, or you
can find other online tools to take your various inputs and figure out the significance of your tests.
WHAT SHOULD YOU TEST?

Here are some of the simple tests that you can create to improve the sales of your Amazon products:

Headline

What text gets the best click through rate? Which keywords drive the best traffic? What
niche should you be targeting? These questions can be addressed by testing your
headline.

Product Description
Your product description is the “fine print” details that your potential buyers will read before they
click the “Buy” button. What description works best for your product?

Main Feature Image

To the time-strapped customer, the image is often the first and sometimes the only thing that
they look at before clicking through on a listing. Is your main product image click-worthy?
You will only know if you are putting your best foot forward by testing different images and
seeing which generates the most clicks.

Text or Graphics on Image

You can do a lot to emphasize a particular product feature or point of differentiation by adding
text to an image. Whether it is a satisfaction guarantee, warrantee, discount, or product detail,
you can test what resonates with your customers by testing this. (Note, it is against Amazon’s
TOS to add an image or text on the main image)

Price

You can maximize your bottom line by testing different price points. What price has the optimal
balance between conversion rate and profit? You will have to test this to find out!

Discounts

Is it worthwhile for you to offer discounts on your products? Or test the difference between your
“List Price” and actual “Price”? Let your customers tell you the answer by running a split
test.
A/B TESTING BEST PRACTICES FOR AMAZON SELLERS:

Don’t Test Too Much at Once

If you test more than one thing at a time, you are running a multivariate test. These are
more complicated to conduct properly. For example, if you change the price, the head-
line, and the image of your Amazon listing, and you find that the variation improved
conversions, how do you know what exactly is responsible for the improvement? It is
hard to know, especially when running tests on Amazon. So it is best to just focus on
one metric at a time to know exactly what you are measuring and improving.

Write Down Your Hypotheses Before the Test:

Before you start your test, just jot down exactly what you are testing, and what
your hypothesis will be. This helps you distill the test down to measure the important
metric you will be tracking, and what metric you will look for when the test is
complete. Then when you have gathered enough data, look at the test results and
compare it to your hypothesis to see if the test was a success.

Be Patient:
It is important that you test a statistically significant data sample before you come to
any conclusion. Split testing will not reveal accurate results immediately. You may have
to devote a few weeks, depending on how many sessions you get. It is important to
remember that a significant data set is more important than a hasty decision based on
incomplete data. Give your tests enough time to give you the answers you need!

Always Be Testing:
If you found a variation that outperforms your control, congratulations! You have incre-
mentally improved the performance of your listing. However, don’t stop there. There is
always room for improvement, and continuing to sharpen your conversions and pro-
cess. Furthermore, things are always changing, so you will have to continue testing the
effectiveness of your changes. What works today may not work tomorrow, for a variety
of reasons beyond your control.
GET STARTED TESTING TODAY
Splitly is a software developed specifically to help Amazon sellers split test listings for improved sales.
Created by Amazon sellers, for Amazon sellers, Splitly is the only way to run accurate split tests for
Amazon products.

It is incredibly simple to set up and run your experiments so you can improve your Amazon listing to-
day.

Run your experiments over time, to get the ex-

act statistical significance you want

Splitly will pull data directly from your Seller account to tell you which variation won!

Get started with a free trial of Splitly to see how A/B testing can improve your Amazon business
now.

START TESTING!
https://app.splitly.com/?ref=36738187