An Analytic Approach: How to Find and Fix Conversion Problems

by Kevin Gao

Last updated on February 14th, 2018

thumbnail series2

This is the final segment in a five-part series of articles on website conversion. How do you solve the biggest problems in website conversion? You read this series!

Part 1: Performance Tuning

Part 2: Logic and Direction

Part 3: The User Interface

Part 4: The Emotional Journey

Today, we wrap up by talking about an Analytic Approach to website conversion: testing. 

Your business website’s main objective is to get visitors to take some sort of action, such as signing up for a newsletter or purchasing a product. This is called a conversion.

Ultimately, conversions drive your business, which means you can optimize your business by optimizing your conversions.

So what do you do when conversions are low?

Hopefully you’ve been following this article series. You’ve already learned tips for improving usability, logic, emotional engagement, and more. It’s time now to focus on the process of optimizing your website for conversion.

I’m coming from one perspective. If you’re reading this article, you:

  • already have an attractive website and are getting a decent amount of traffic
  • suspect your conversion rate can be improved
  • are willing to confirm your suspicion through experimentation

Keep reading, and I’ll tell you how it’s done. Ready to start? Let’s go.

The Optimization Process

To optimize anything, you need to conduct a scientific experiment, carefully analyze the results, select the optimal result, and repeat. If everything is done correctly, the results get better and better over time.

This concept applies very nicely to website design.

A good website design is one that delivers consistently high conversion rates. But even the best websites can increase conversions. You simply need to know how.

The key is to avoid arbitrary decisions. For example, let’s say you need to choose between two different photos as part of a car advertisement.

One photo features an attractive woman leaning against the driver’s door with a beach in the background. The other photo features a similarly attractive woman in a similar pose within a country setting.

 woman beach woman country

Which photo will generate the most conversions?

Either photo would probably work well, but which one works best? Without experimentation, the decision seems arbitrary. But with experimentation, you can test both photos and then select the one that provides the higher conversion rate.

Action – Meet with your web designer and create a short list of isolated decisions that can be tested. For example, does the “Call to Action” button work best along the right margin or the left margin? And should it be yellow or red?

Tip – There’s a good chance that your web team already has a list of hotly debated design issues. This is a good starting point for running your “split” and “multivariate” tests, as described in the next sections.

Split Testing

Also known as A/B testing, split testing seeks to optimize results when trying to decide between two isolated parameters. The example above, with two photos, demonstrates this concept—you must choose between one of two photos.

Here’s how the split test works:

  1. Decide what constitutes a “conversion.” You may decide that a conversion is simply a click on a button to get more information, the downloading of some content, or placing an item in a shopping cart.
  2. Determine the sample size. In other words, decide the number of visitors needed for your experiment. The larger the sample size, the more accurate your results will become.
  3. Create a separate Web page for each of the two variations. Implement special code on each page so that the corresponding conversions can be separately tracked. Note that you must keep track of which page is your “primary” page, so after the sample size has been exhausted (i.e.; the experiment has run its course), the website will fall back to the primary page.
  4. Upload both pages to your site.
  5. Implement special code so that each visitor sees only one of the two pages. Use a cookie to ensure that each visitor sees the same page each time he visits your site.
  6. Track the total number of visitors to the pages, along with the number of conversions that occurred for each.
  7.  Ensure the server falls back to the “primary” page after the sample size has been exhausted. At that point, stop taking data.

After running the above experiment, you analyze the results. If it appears that one choice is clearly better than the other, you should change the “primary” page accordingly, thereby optimizing your site for conversion.

For example, the photo ads above yield the following conversion rates:

Split Testing

The beach photo is the winner, as it shows the higher conversion rate, even after accounting for the +/- variation. The website should thus be changed to implement the beach photo.

Action – Review the list of isolated decisions created in the previous section and look for decisions where there can only be one of two outcomes. Prioritize this list based on the highest possible conversion rate. Start working through the list, one at a time, using the experimental process highlighted above.

Tip – Pay attention to the “confidence interval.” This number indicates the variation in your results that must be taken into account. Generally, the variation gets smaller as your sample size gets larger.

Multivariate Testing

The concept of multivariate testing is similar to split testing, with the exception that it compares multiple pages against each other. Multivariate testing thus allows you to address issues in scope, rather than simple binary decisions.

For example, as winter approaches, you may want to incorporate a winter theme on your site. Should you implement something minimal like hanging an icicle from your logo? Or should you go further and simulate snowflakes dropping across your entire page along with the icicle?

Your marketing team may come up with dozens of creative ideas. You collect the ideas into groups and create pages based on the groups. Essentially, you’re testing groups of multiple changes on a page.

The step-by-step experimentation process is similar to split testing, except you’re loading multiple pages into the test, rather than just two.

Be aware, the code for multivariate testing is a bit more complicated because you are now splitting each of your visitors into one of many pages and keeping track of conversion rates on each page.

Also, keep in mind that the original page must be part of the test. It is the baseline that is used to compare the conversion rates.

When the experiment has concluded, you may end up with something that looks like this:

Multivariate Testing

At first glance, it would appear that the Group 3 page, with 29.19% improvement, produced the most superior conversion rate. But it’s important to exercise caution. The variance (the +/- after the conversion rate) is still quite wide.

The original page expressed a conversion of 6.20% plus or minus 3.5%. Despite the fact that Group 3 looks vastly superior, the conversion rate variance from both the original page and Group 3 overlap.

Let’s put that into a chart, and you can see what I mean:

Original Page: 6.20 – 3.5 = 2.7 or 6.20 + 3.5 = 9.70

Group 3 Page: 8.01 – 3.2 = 4.81 or 8.01 + 3.2 = 11.21

conversion rate data

The two bars overlap, which means you don’t have enough statistical confidence that Group 3 is indeed better than the original.

The advice here would be to increase the sample size to decrease the variance, and therefore determine whether the changes in Group 3 really are superior.

After testing all the groups and establishing conversion rates that differ by more than their tolerance levels, you would simply implement the page design with the highest conversion rate. You could then create more groups and repeat the experiment.

Action – Review the list of isolated decisions created in the first section of this article and look for decisions where there are many possible changes. These types of changes generally pertain to the scope of the page (e.g., the template) rather than simple isolated “this-or-that” decisions.

Start working through the list and create a set of changed pages for each. Then load these pages into the system, using the experimental process similar to the description for split testing.

Tip – You want good confidence intervals, so you’ll have to take a lot of samples; however, you will want to limit the number of experiments to keep from confusing your visitors. Therefore, consider selecting only a small percent of your visitors, say 10%, and experiment on them.


There are a lot of factors that affect website conversion, and it can be difficult to decide which are superior. You can use split testing to optimize between two isolated parameters. And you can use multivariate testing to decide between groups of parameters.

In either case, you conduct a scientific experiment on your website visitors to determine which changes yield superior conversion rates.

As a result, your decisions are never based on personal preference or the opinions of people inside your organization. By testing your options, you let visitors tell you which page elements are most persuasive.

In this case, the customer is always right.

Photo Credits: Ivan Mikhaylov, Vladimir Voronin

No Comments


Get updates on new articles, webinars and other opportunities:

Kevin Gao

Kevin Gao is the founder and CEO of Comm100, a leading provider of live chat software for business. As a software developer as well as a small business expert, he’s always ambitious to revolutionize the way of online customer service and communication. Follow Kevin on Google+ to find more about him.


Comment Policy

Please join the conversation! We like long and thoughtful communication.
Abrupt comments and gibberish will not be approved. Please, only use your real name, not your business name or keywords. We rarely allow links in your comment.
Finally, please use your favorite personal social media profile for the website field.


Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Show Me My Heatmap

Ah, @CrazyEgg I really do love you! So useful evaluating how users are interacting with all aspects of our redesign

Mike Halligan


What makes people leave your website?