Case Study

What Should I A/B Test? An Ecommerce Homepage Analysis

Disclosure: Our content is reader-supported, which means we earn commissions from links on Crazy Egg. Commissions do not affect our editorial evaluations or opinions.
ecommerce homepage analysis

There are four key types – or groups – of data that are used for conversion rate optimization (CRO):

There are four key types – or groups – of data that we use for conversion rate optimization (CRO):

We know that data (Analytics) tells us what is happening.

We combine data and input from sources like Google Analytics (Analytics), Crazy Egg (Analytics, Heatmaps, User Testing), and SurveyMonkey (Analytics, User Testing) to help us understand why it is happening.

We look for patterns that help us develop insights on visitor behavior.

Then we use the data to inform our hypotheses about why particular behaviors are occurring and how we can impact those behaviors.

This strategy helps us design on-site experiments and tests (A/B Testing) that give us better insight into consumer behavior.

All of this together makes up CRO — an iterative process of small incremental changes that add up to a more engaging site, tailored to consumer behavior.

The Importance of Website Analytics in A/B Testing

A lot of marketers talk about A/B testing best practices, but the truth is that what works for one company won’t necessarily work for another.

To minimize risk and set your A/B tests up for success, my best advice is to run an analysis of your most valuable landing pages first.

That way you can easily identify what will have the most impact on your business growth, and any design changes you make will be backed by actual visitor data.  

Often, the most valuable page for any business is the homepage, since it’s where most traffic is directed via search, social and paid ads.

The brave team at recently volunteered their homepage for a live analysis with my CRO agency, The Good, and Crazy Egg’s Customer Success lead JL Neilsen.

If you missed it, here’s the on-demand recording:

I’m also going to share the highlights with you below.

Landing Page Analysis:

Today we’re going to start analyzing the top of Tommie Copper’s homepage, and work our way down to the bottom. We’ll be using Crazy Egg’s Snapshot reports (heatmap, scrollmap, confetti, list and overlay) to gather valuable data points. 

As I go along pointing out opportunities for optimization, remember that you should think of all my suggestions as “suggestions for A/B tests that should be run on the site.”

So, here we have Tommie Copper’s homepage. Before we start looking at the visitor behavior reports, I’m going to ask you use a critical eye, take a moment, and review it briefly yourself.

  • What do you immediately notice? 
  • Where is your eye drawn to?

Looking at your website through the lens of a first-time visitor and trying to accomplish certain tasks always a good place to start when you’re running a CRO exercise. Now, let’s dive in. 

Homepage Heatmap

If you’re not familiar with heatmapping, all you need to know is that it goes from hot to cold:

  • Red is where click activity is highest
  • Then it cools off from there… so red to yellow to green to blue
  • Lack of color means a lack of engagement.

OK, so let’s take a quick look at this heatmap. What do we immediately see here?

Email Popups

I’m on a crusade to rid the internet of these. I hear it all the time: “But they work!”

But do they? 

Email pop-ups, or overlays, are a huge barrier to the user flow.

Imagine walking into a store and the salesperson jumps in front of you with a clipboard demanding your email. That’s not cool, right? You’d likely turn around and leave, right??

We’ve been helping brands convert higher on their sites for 10 years now, and we’ve only seen two reasons consumers are at your site:

  1. To conduct research and understand if your product or service is going to solve their pain or need
  1. To convert, or complete a purchase as quickly and easily as possible, if you can indeed solve their need

Interactions like email pop-ups that disrupt these two goals are only going to turn people away.

In fact, if we look at Crazy Egg’s Overlay report, we can see 1,983 people have clicked on that big X to close the form and get on with their visit. 

My advice is to try not doing a pop-up at all, and instead baking these forms into the page content where users won’t be disrupted.

If you insist on using an email pop-up, though, here are a few tips.

CRO Suggestions:

First, visitors will be looking for privacy information:

  • if I give you my email, how are you going to use it? State a clear privacy statement near the sign-up field.

Second, visitors will want to know how often you will you use their information:

  • How many emails are they committing to receiving?
  • Can they unsubscribe with one click at any time?

Third, they’ll want to know if they can sign up later, perhaps as part of the checkout process, if they don’t want to do it now. Reassure the visitor that the opportunity isn’t just gone if they close the pop-up now.

Next, let’s move on one of the most crucial elements on any ecommerce website. 

Navigation Bar

The navigation bar is well done here!

You can see plenty of interaction on the heatmap, which is a great sign. Overall, this nav bar is compact and helpful to the consumer. We’ve found that a main site navigation should have no more than 5 items, so the 4 listed here is great.

The prominent search bar is also excellent. 

CRO Suggestion: Visitors who use search will convert the highest on your site, because they know what they want, and are looking to go directly to that product. So, keep in mind how important it is to optimize your Search Results pages.

Image Carousel

It is a bit hard to see behind the pop-up, but there is an Auto-Rotating Image Carousel in the background. You can see from the heatmap that a lot of people have clicked on the arrows to move the images forward or backwards. 

I can just tell you flat out – these kill conversions. 

I understand why teams love these as an option; carousels communicate so much info in a small area. But let me tell you in our experience why they don’t work or how they fall short:

  1. Distracting – They are a distraction because they are moving at seemingly random intervals as the visitor is looking at other content on the page.
  2. Frustrating – They are frustrating because as the visitor is trying to read or look at the image, it often will rotate to the next one before they are ready. Trying to click on a CTA in a rotating image can compound this frustration.
  3. Evidence – The data clearly shows that they are ineffective. In a research report by University of Notre Dame only 1% clicked the feature. Of those, 89% were the first position. 1% of clicks for the most significant object on the home page? This means anything past the first image is unlikely to get seen at all (and certainly not interacted with). Note: Our data across hundreds of sites over 10 years replicates these findings.

Product Tiles

Despite all the click engagement you can see in the heatmap, there is one big missed opportunity when we get to the meat of this page. 

You can see from looking at all the gray circles on Crazy Egg’s Overlay Report that a lot of these are “dead” clicks — meaning, people are clicking in areas that aren’t linked to anything. 

In this example, you can see that 45 people clicked near the “Shop Now” CTA, but that area is a dead zone. One easy fix is to link all the tiles to their respective collection landing pages. 


Touts – like this one here that says “Up to 40%” off – tend to perform really well. 

You can see how that promotion is influencing clicks on the “Shop Now” CTA for Compression Tights. When you look at the Overlay report below, it’s clear that the “Up to 40% off” callout is contributing to more clicks than the Compression Tops  tile above it.

Touts, used strategically and thoughtfully, will tend to be more effective than Text or Text alone.

Bestselling Items

Scrolling down a little more we encounter the Bestselling items. You can clearly see from Crazy Egg’s scrollmap that engagement has dropped dramatically here. 

This is very likely because each product is missing a clear CTA like the tiles above have.

One’s instinct may be to push for sales directly here. However, to increase engagement and time spent in this area, I would strongly suggest testing the addition of low intent CTA buttons here such as:

  • View More
  • Learn More
  • View Details

Leading with “Buy Now” may actually prevent clicks and lower engagement, as opposed to low intent CTAs which can help bring your audience along the path to purchase.

Something to consider testing is removing the star ratings. Social proof is compelling, but we’ve already indicated clearly that these are best sellers; at this point the Star Ratings may be redundant. 


Finally, let’s take a look at the Footer. The first thing I notice here is that Contact Information is missing from the bottom right-hand corner of the Footer. In our testing we see that nothing increases trust more than having contact information clearly showing in the footer. 

Location matters because the first place consumers look for contact information is in the bottom-right corner of the site.

To be clear: yes, there is a “Contact” link here in footer – in fact, you can see how much interaction it is getting! But it’s not in the optimal location based on user expectations.

You can see from Crazy Egg’s Confetti report that the majority of people clicking on the Contact link are new visitors; making it doubly important that this link be placed more prominently so people can engage with the brand and become loyal customers. 

Next, I’m concerned that there is only one option for the consumer to continue down the shopping path if they reach the footer. 

If someone is going to scroll all the way down and read the entire home page, they are interested in your products — which means you should give them clearer pathways back into your funnel. 

Ideally you would repeat the main navigation right here in the footer as a new column. Don’t hide it. Make sure the navigation items you felt deserved premium placement atop your site are now clearly called out (in this case, I would find a more prominent way to feature the word “Outlet”).


Okay, we made it! There were a lot of good learnings there –– thanks so much to Tommie Copper for allowing everyone to learn from their assessment.

One key takeaway here is that today is just the tip of the iceberg when it comes to CRO.

We only had time to take a brief look at some of the challenges on the pages we reviewed.

CRO itself goes way way deeper, and we’d want to collect all the types of data we discussed, as well as run A/B tests based on all the observations and suggestions we made today.

getting started guide to cro

Author Bio

Jon MacDonald The Good

Jon MacDonald, CEO of The Good

The Good has been helping ecommerce companies increase sales for about 10 years. Our services convert more of a company’s existing website traffic into buyers. We do this through conversion rate optimization –– including A/B testing and data science – to improve usability.

Make your website better. Instantly.

Over 300,000 websites use Crazy Egg to improve what's working, fix what isn't and test new ideas.

Free 30-day Trial