Editor’s Note: This is the second article in a 4 part series.
- Part 1 – A Beginner’s Guide to A/B Testing with Crazy Egg
- Part 3 – Best Practices and Pro Tips for Using An A/B Testing Tool
- Part 4 – How to Understand and Act On Your A/B Test Results
One misstep we often see companies make:
Investing in a massive website redesign without validating the changes they’re making by looking at their customer data first.
The downside of this mistake is massive; wasted time and money, to be sure, but also, this is a huge missed opportunity for making positive and impactful website updates!
Before you launch an A/B test, it’s extremely important that you look at your website visitor data. This will help you figure out which changes will have the most positive impact on the browsing experience you are providing your leads and customers.
In this post, we’ll teach you how to:
- Set up visitor reports on your website
- Run a site analysis
- Share your insights with team members and stakeholders
We’ve got a lot of ground to cover, so let’s get started!
Note: Check out Part 1 of our ongoing A/B Testing series – A Beginner’s Guide to A/B Testing With Crazy Egg
If you’re new to the world of Crazy Egg or want to double check your setup, read on. A “Snapshot” is our term for a digital capture of your webpage.
This allows us to track visitor clicks, scrolls, and pauses on your site.
You can create one or more Snapshots at a time, and customize each Snapshot to suit your particular website.
In the example above, I’ve created a Snapshot because I’m interested in redesigning my homepage (https://www.quicksprout.com)
I have chosen to create a separate Snapshot for each of the different devices visitors use to navigate my homepage:
I chose this option because I want to check if my website is truly responsive, and to find out whether visitors behave differently based on their device type.
When we take a look at the heatmap Snapshot, you can see there is in fact a significant difference!
Pro-tip: If your homepage or landing page contains pop-ups, you can create one Snapshot that shows pop-ups (so you can see how people interact with them) and another Snapshot of the exact same page that omits pop-ups (so you can see how visitors interact with your CTAs instead).
Use Crazy Egg’s Snapshots To Run A Site Analysis
Once we’ve started tracking your visitor clicks, it’s time to start analyzing your Snapshot Reports.
To give you a brief overview, these are some of the signs that you should look for:
- Are people clicking where you’re expecting them to click?
- Are you leaving escape hatches for people to exit the page because they’re getting distracted by too many navigation elements?
- Are people looking for more education?
- Are there dead clicks (meaning, people are clicking on non-interactive elements)?
- Should you add social proof that shows you are trustworthy?
You can find the answers to each of these questions by going through our five Snapshot reports. Think of this as your diagnostics phase; you’re trying to figure out what is stopping your visitors from taking the action that you want them to take.
From there, you can come up with an AB test based on the change you’d like to make your site.
- Move the CTA or change the color to see if it attracts more attention
- Decrease the number of elements in your top navigation
- Create a video explainer and put it next to the CTA to see if that improves signups or trials
- Make elements that are being “dead clicked” interactive, and see if that boosts conversions
- Add in testimonials or logos of people or companies who have purchased from your business
But we’re getting a little ahead of ourselves. First, we’ll explore how you can conduct visitor research, based on real data and site interactions.
Here is one method you can use to run a website analysis, leveraging Crazy Egg’s Snapshot Reports:
1. Start with the Heatmap
Let’s go back to the Snapshot I created on my homepage. In this instance, my primary goal is to boost URL submissions on the primary CTA.
By looking at the heatmap above, I can see that the URL submission field and the blue CTA are receiving comparable amounts of clicks — but the real hotspot is the “Blog” link in the top navigation (you can tell it’s the most popular because it’s bright yellow).
A/B Tests #1 and 2: Off the bat, an A/B test that I can launch quickly would be to reduce the prominence of the Blog link (either by moving it down the page, or adding it to a dropdown menu) since I know that people are clicking on it rather than submitting their URLs. A secondary A/B test could be changing the color of the blue CTA to see if that attracts more attention.
2. Check out the Scrollmap
One major benefit of a Scrollmap is being able to see whether your primary CTA is positioned where the maximum amount of impressions occurs (this is calculated when a visitor stops scrolling).
A/B Test #3: If my homepage CTA is in an area that receives fewer impressions (meaning, people aren’t scrolling down far enough to see it), I should run an AB test to see if moving the CTA further up the page helps boost my conversion rate.
However, in this instance the CTA is positioned fine, so I likely won’t experiment with moving the CTA this time around.
3. Head over to the Confetti Report
While the Confetti report gives me 22 different filters that I can use to get to know my customer segments — including referral source and country — I’m going to focus on the “new vs returning” filter for this particular example.
A/B Test #4: I’m thinking about running an A/B test where I add social proof or some sort of explainer video near the CTA on my homepage, so that people who are new to the page are motivated to click.
The Confetti report shows me that I have much more new visitors than return visitors. This means it’s a good idea to provide more information or education around what my company provides.
I will likely run this as a third AB test, after I experiment with the Blog link in the nav bar and the color of the CTA button next to the URL submission field.
Pro Tip: It’s best to only make one major update to a page at a time so you know exactly what it is that is impacting your visitors’ decisions.
4. Move on over to the List Report
From the Confetti Report, I like to head over to the List Report to get a bird’s eye view of what elements are being clicked, and what the percentage of those clicks is in relation to all the clicks I’ve gotten on the page.
I’m curious about what the click difference is between the URL submission form field and the CTA button. There seems to be a dropoff of about 17%, so this is another indicator that changing the button color could have a positive impact on my conversion rate.
Another goal going into the List Report is to see how many elements in the top ten are not interactive (read: what content could benefit from being linked).
A/B Test #5: In this case, it looks like the headline of the homepage is receiving enough clicks that I should consider either creating more space between the CTA and the headline, or making the Headline clickable.
5. End on the Overlay Report
You can click on any of the visible elements on the List report and get taken to that exact element on the Overlay Report. Once there, you can filter clicks by the same 22 different filters that you had on the Confetti Report.
In this instance, I’ve clicked on the “Grow your business, faster” headline because I wanted to double check which element it was on the page.
I can also see areas that are getting “dead clicks” (indicated by those gray plus signs) when I toggle the “Other” or “Both” tabs.
This lets me see whether there are opportunities to turn those dead click areas into interactive elements.
During the next round of analysis, I can return to the Overlay and Confetti reports to find out a little more about the types of people who are clicking where I want them to click, and what I could do to improve that number even more.
Using Crazy Egg’s Visitor Recordings to Fill in Any Blanks
Now that I’ve got a handful of ideas about what to change on the homepage, I’ll head over to Visitor Recordings to double check if there is anything I’m missing.
This session replay feature allows me to watch people navigating my site in real time, which makes it a great tool for validating the observations I made during my Snapshot Report analysis.
Before launching an A/B test, it’s helpful to watch at least 15 to 20 minutes worth of recordings to get a solid idea of how changes might impact your visitors.
If you’d like to learn more about using Visitor Recordings as a part of your CRO routine, check out these two articles:
- Visitor Recordings Part One: Seeing Value And Overcoming Obstacles
- Visitor Recordings Part Two: Knowing What To Look For
Pro tip: To narrow down the amount of recordings that you need to watch to get valuable insights, you can use filters like Visited Page and Referrer. “Ad Campaign” is another great filter that lets you see how your paid spend visitors behave on your site.
The Buy In: How to Convince Your Team
This last step in the process of planning your A/B test is optional, but it’s an important one: getting your team on board with your optimization ideas.
Share A Snapshot Report and Add a Note
Depending on the number of stakeholders at your company, you might need to convince other colleagues across teams that your A/B test is based on real visitor data.
You can easily do this by sharing a Snapshot (which includes the heatmap, scrollmap, confetti, overlay and list reports that we walked through earlier) and adding a note about what you’ve observed.
Share A Visitor Recording and Add a Note
Similarly, when you see interesting or puzzling behavior in a visitor Recording, it’s helpful to be able to share the exact minute and action that you want your teammates to see as well.
Sharing a recording with a note is a great way to bring static website visitor data to life, and get team buy-in on your A/B tests. It’s also an easy way to share any site glitches or unexpected browser issues with your development team.
Ready to Launch Your A/B Test?
Now that you know how to come up with great ideas for improving your conversion rates by using actual visitor data, it’s time to launch an A/B test.
If you’re still hesitant about whether your ideas will actually make a difference to your goals, keep in mind that Crazy Egg’s Multi-Armed Bandit approach to A/B testing is low-risk.
As we covered in the first article of this series, while we begin with a traditional split of 50/50, as click data rolls in we gradually direct traffic to the higher performing variant. That way, you won’t lose out on conversions while we determine a winner.
In the next article of this series, I’ll walk you through the steps of creating an A/B test, and share best practices for your first time out (or how to sanity check your current setup).