Amazon A/B Testing: Step-by-Step Guide

Jennifer Robinson -

Aloha, Amazon Seller!

Follow our article on what А/В testing is and you can learn how much of a difference can split testing make for your business, and why it’s a seller’s secret weapon.

Amazon is a marketplace that gained tremendous success due to innovation and improvement, sometimes even giving headaches to the merchants because of constant changes. The same is encouraged for the sellers who are doing their business on Amazon: they are expected to be open to implementing efficient scientific methods to increase sales and enhance customer experience.

One of such online shopping enhancement tools is A/B testing. It is aimed at improving your listings’ and website’s conversion rate and conversion funnel, return on ad spend, and reducing cart abandonment rate, thus helping generate more revenue.

However, it is important to use the split test correctly, avoid “chaotic” A/B testing, and perform proper split test analysis. Also, it is important to understand, that you need to do a series of tests for the best result. Amazon’s Manage Your Experiments tool can assist you with this.

What Is Split Testing?

Split testing, also known as Amazon A/B testing is an experimental way to find out which design, content, or functionality works best for your shoppers and makes customer journey easy and efficient.

Since on Amazon there’s no way you can just copy somebody else’s listing design or content, as it might not work in your case, you need to optimize your own listing in line with consumer behavior best practices.

But how do you know what your shoppers like or dislike? The only way to determine what content, layout, or design, works best for your buyers is to test a variation of your page (or element on a page) and experimentally discover which one convinces them to make a purchase decision.

How Does Amazon Split Work?

The principle behind split testing is that you choose one of the website elements you want to optimize and run two or more of its versions simultaneously to analyze customer response.

Split Amazon works differently unless you are enrolled in Amazon’s Brand Registry. If you are not registered, you need to run each version of your website successively, for a specific period of time.

However, enrollment in Brand Registry allows merchants for access to Amazon’s Manage Your Experiments programming tool, enabling them to run a number of their listing’s versions simultaneously.

Efficient A/B testing examples involve repeatedly testing improvements until you get the optimized listing version based on measuring such metrics as sessions, conversions, conversion rate, and profit.

For A/B test choose a website element you want to optimize and run two of its versions simultaneously to analyze customer response

Why Should You Do A/B Testing?

As a seller, you know that a happy customer is the greatest ad and a perfect brand ambassador. If you know the basics of your customer behavior, you know what you can enhance to increase sales.

There are at least 5 reasons why you should do an A/B Testing:

  • It helps increase visibility;
  • It helps optimize customer journey;
  • It helps improve customers’ experience: happy customer leaves a positive review;
  • It helps generate customer loyalty;
  • It helps drive conversions, thus generating more revenue.

Where Can You Use A/B Testing?

There are a lot of things you can split test on Amazon.

Picture/A+ Content

A picture is the most important element of your listing. Bright professional pictures increase visibility and make your listing stand out. Also, it gives shoppers an idea about the product since they cannot see it live. Amazon Standards for Product Images give you good tips as to how it should look like.

Amazon Site Standards for Product Images

  • For images named by product identifier without a variant code or named with the MAIN variant, and display as the main image on the product detail page, Amazon maintains the following site product image standards:
  • The image must be the cover art or a professional photograph of the product being sold. Drawings or illustrations of the product are not allowed.
  • The image must not contain gratuitous or confusing additional objects.
  • The image must be in focus, professionally lit and photographed or scanned, with realistic color, and smooth edges.
  • Books, Music, and Video/DVD images should be the front cover art, and fill 100% of the image frame. Jewel cases, promotional stickers, and cellophane are not allowed.
  • All other products should fill 85% or more of the image frame.
  • The full product must be in the frame.
  • Backgrounds must be pure white (RGB 255,255,255).
  • The image must not contain additional text, graphics, or inset images.
  • Pornographic and offensive materials are not allowed.

Test different images to select the most compelling one.

Title

The title is the second thing about your product that appears in Amazon’s search results. A messy unclear headline will mislead the customer and cause a lot of negative reviews. A title should clearly state the most important information related to your product, like functionality, size, etc. Amazon’s title guidelines give you good tips as to how it should look like.

Amazon’s Product title requirements

  • Titles must follow the recommended length of your product category characters, including spaces.
  • Titles must not contain promotional phrases, such as "free shipping", "100% quality guaranteed".
  • Titles must not contain characters for decoration, such as ~ ! * $ ? _ ~ { } # < > | * ; ^ ¬ ¦
  • Titles must contain product-identifying information, such as "hiking boots" or "umbrella".
  • Titles should be concise. We recommend fewer than 80 characters.
  • Don't use ALL CAPS.
  • Capitalize the first letter of each word except for prepositions (in, on, over, with), conjunctions (and, or, for), or articles (the, a, an).
  • Use numerals: "2" instead of "two".
  • Don't use non-language ASCII characters such as Æ, ©, or ®.
  • Titles should contain the minimal information needed to identify the item and nothing more.
  • Don't use subjective commentary, such as "Hot Item" or "Best Seller".
  • Titles can include necessary punctuation, like hyphens (-), forward slashes (/), commas (,), ampersands (&), and periods (.).
  • Titles can abbreviate measurements, such as "cm", "oz", "in", and "kg".
  • Don't include your merchant name in titles.
  • Size and color variations should be included in titles for child ASINs, not the main title (see below). Test out various versions of your title to see what generates more conversions.

Bullet Points

The bullet points are used to reflect details about your product’s features and benefits in a clear and short way. They should inform buyers what exactly your product is, and show its advantages over your competitor’s. Don’t forget about your keywords as they are extremely important for your visibility, but don’t over-spam.

Split test different lengths of the bullet points, and the order of the item’s features.

Description

Description generally includes the same information as bullet points, but it is more detailed and allows you to include more keywords to drive more conversions. You can add additional information you consider important.

A/B test different versions of description and keyword combinations.

Pricing Strategy

The best price is the one matching customer value. Therefore, private label products, for instance, are never priced too low. When you’ve outlined your Amazon FBA strategy, you should have already decided what is the customer value of your product. If you overprice your product, shoppers will go and buy from your competitors. If you underprice, chances are that you run into two very unpleasant consequences. One is that you don’t get back the money you invested because your expenses are not covered. Two is that you might initiate a price war with the competitors, which is a rather expensive and dangerous thing unless it is a strategy in itself aimed at destroying the competition.

To test in practice the price you figured out a good way to do it is the A/B pricing test. This is how you find out if your price is competitive and prove that you’ve calculated everything the right way.

Perform an A/B pricing test for different price levels and let them sit there for some time. Afterward, analyze the key metrics and pick the most successful one.

PPC campaign

Amazon sellers can use Amazon A/B testing to evaluate different versions of their product listings, and figure out which listing they should use to run their Sponsored Product Ads.

With off-Amazon tools, you can also A/B test your frontend and backend search terms, discounts, coupons, etc.

Benefits and Challenges of A/B Testing

Benefits

  • Improved return on ad spend (ROAS);
  • Reduced customer acquisition cost;
  • Increased lifetime customer value (LTV);
  • Increased email signup conversion rate;
  • Increased average order value (AOV);
  • Reduced cart abandonment rate;
  • Increased e-commerce conversion rate (CR).

Challenges

  • It should be done properly (see the tips below);
  • It’s a series of tests, and requires stats analysis;
  • Traffic is required.
Amazon sellers can use Amazon A/B testing to evaluate different versions of their product listings

A/B testing Tips

Before starting a split test, you need to elaborate a roadmap, helping you properly set its parameters.

  • What Should You Know Before Doing Your A/B Test?
  • Why are you testing an element?
  • What do you want to know?
  • What is the length of the test?
  • What metric do we use to measure and judge results?
  • Which variations are best in this test?
  • What action will you take based on the test results?

A/B Testing Pro Tips:

  • Make sure you are testing a single variable at a time;
  • Include highly converting keywords;
  • Run your A / B test at least 4 weeks at a time;
  • Analyze your A/B testing history;
  • Do a series of tests for the best result;
  • Minimize the influence of external factors, like Black Friday.

How to Do A/B testing?

There are four options to run split tests: manual, Amazon’s Manage Your Experiments Tool, creating a poll off-Amazon, and A/B testing seller tools.

Manual A/B testing

Performing A/B testing manually is a complicated and time-consuming process. As said, you need to test only one element of your listing at a time. To track changes to your metrics, you will need to check them at Amazon’s business reports within your Seller Central account.

Here’s how you do it:

Step 1. Log in to your Seller Central account, navigate to Reports > Business Reports.

Step 2. On the left-hand side, click on “Detail Page Sales and Traffic By Child Item.”

In the report, you will see all metrics for each product and monitor how your products are performing.

Step 4. In a spreadsheet, make note of the start and end dates of each version’s test.

Step 5. Check your sessions, conversion rates, and units ordered before and after each test, and input the data on your spreadsheet.

Step 6. Analyze the data to see which version performed better.

Manage Your Experiments Tool

Enrollment in Brand Registry allows brand owners access to Amazon’s Manage Your Experiments tool, enabling them to run a number of their listing’s versions simultaneously, which makes the whole process more efficient and less time-consuming.

Here’s how you do it:

Step 1. Navigate to the Brands tab then click on Manage Experiments.

Step 2. Click on “Create a New Experiment”.

Step 3. Select between A+ Content, Title, or Main Image.

Step 4. Choose your product, enter the product data you want to test, and input the test duration.

Step 5. Click on “Schedule Experiment.”

Step 6. View the results of the A/B test every week.

Create a Poll

You can do A/B testing off Amazon, conducting a poll with your potential customers in social media, like Instagram, Facebook, YouTube, Tiktok. You can create a carousel post with two main image options. Ask people to comment “A” or “B” option.

A/B testing seller tools

There are a lot of third-party A/B testing seller tools offering a lot of options. You can select any of those available on the market.

How to Check Amazon Split Testing Results?

The results of Amazon listing testing can be detected by measuring the metrics of the A and B versions of your listing. You’ll need to wait for 4 to 10 weeks until you will see the result. If your traffic and conversions have grown on one of the samples, you’re there.

Where Can You Check Your Listing Metrics?

To find out how much your listing metrics have changed, you can use your Seller Central business reports and specialized seller tools allowing you to track your product performance and KPIs.

Checking your A and B samples in the Seller Central:

Step 1. Log in to your Seller Central account, navigate to Reports > Business Reports.

Step 2. Click “Detail Page Sales and Traffic By Child Item.”

Step 3. From the report, find out Sessions, Buy Box Percentage, Units Ordered, Unit Session Percentage (conversion rate), and Product Sales. Put it down in a spreadsheet.

Step 4. From time to time, check how A and B samples behave.

Step 5. At the end of the A/B test, analyze the data and select the best-performing variant.

Note: At the beginning of your experiment, record all initial listing parameters, to be able to further analyze the data and track any changes.