Get updates on new articles, webinars and other opportunities:

How to Choose the Right Testing Software For Your Business

by Jacob McMillen

If you’ve spent any amount of time on The Daily Egg or similarly focused blogs, you know that thorough testing is paramount to your web business’ success.

Every customer base is unique, and winning marketing victories depends on understanding the users visiting your website each and every day. You might be intuitive (or lucky) enough to correctly read your customers on any given campaign, but over the long run, success WILL depend on data analysis.

Despite the hundreds of voices claiming that marketing is an art, you’ve heard of Neil Patel, Rand Fishkin, Avinash Kaushik, Danny Sullivan, and Brian Clark for one reason: They understand marketing is a science.

Unfortunately for the the small business owner, data analysis is decidedly technical. Simply reading a few blogs won’t be enough to turn data into dollars. You’ll need a handful of powerful tools at your disposable.

Fortunately, there are an abundance of such tools available.
If you have a marketing team ready to pounce on this, these programs will provide an effective setup for efficient testing. If the burden falls on you, the business owner, there will be a slight learning curve. But don’t worry, each of the programs listed here are highly successful, fully supported, and come with a dedicated team invested in making your experience flawless.

Keep reading for our review of six A/B testing software.

1. Google Analytics

google analytics review

As the only free option on our list, Google Analytics is the baseline of all Internet data analysis. It offers users the rare opportunity to draw directly from Google’s extensive data pool. If you’ve never used Google Analytics before, or if you’ve wandered around the dashboard a few times without really grasping its potential, here’s a great place to start.

Once you’ve learned the basics, it’s time to use this new-found data to drive revenue for your business. This requires actionable steps, and you can begin here.

So what about A/B testing?

Analytics offers a feature called “Content Experiments,” which allows users to split-test page variants within the program. It’s a useful feature with advanced reporting options but unfortunately lacks a visual editor, posing a major stumbling block for many would-be users.

The main drawback to Google Analytics is the steep learning curve.

There’s a wealth of information there, but you’ll need to be a pretty proficient analyst to take advantage. Advanced use of Google Analytics should be an aspiration of every business owner, but it’s definitely not the easiest place to start.

The Pros

  • It’s free!
  • Simple interface
  • “Goals” feature allows for easy tracking of business targets
  • Offers quicker, more efficient multi-armed bandit testing
  • Directly integrated with other analysis features and huge data pool

The Cons

  • Raw data – interpretation and use are on you
  • Lacks multivariate testing and other testing options offered by premium competitors
  • Difficult to use on dynamic pages
  • Lacks a visual-based editor
  • Unable to integrate additional data sources

2. Optimizely

optimizely screen cap

Optimizely is a great first stop for business owners wanting to start testing. Installation is remarkably simple, and the WYSIWYG interface is highly intuitive, even for newbies. With plans starting at $17, Optimizely is a very cost-efficient entry point.

But don’t mistake simplicity and budget pricing for an inferior product. Optimizely’s testing software is good. Very good. With split, multivariate, and mobile testing options, you’ll be teaching data anlysis by the time you’ve outgrown this product’s utility.

Plus, as your company grows and you become more familiar with testing, advanced features at the higher price point allow for continued use without needing to learn completely new software.

If you’re starting completely from scratch, Optimizely might not be your best option. You’ll need passable landing pages to test in the first place, and you are entirely responsible for creating your own A/B tests. This is a pretty big prerequisite for someone whose marketing knowledge consists of nothing more than a few blog post reads.

The Pros

  • Cost-efficient premium option for entry-level users
  • Easy, intuitive interface
  • Effective A/B, multivariate, and mobile-focused testing options
  • Solid basic offering with quality advanced features at higher price point

The Cons

  • Lacks outside data integration
  • No algorithm alternatives for judging results, such as bandit or adaptive testing
  • Multivariate testing requires expensive Gold membership ($359 per month)

2nd Opinion

“To start, Optimizely is very competitively priced and dead simple to set-up and install, so the bar to start getting a return on your investment starts off low. From there, it’s up to you. What ideas do you have to improve your digital business? Optimizely is a tool that will help you test and measure them, and quantifiably tell you if they work or not. Your return will be as good as your testing ideas.”

Ryan Garner, Co-founder Clearhead

3. Visual Website Optimizer

visual website optimizer screen

Visual Website Optimizer (VWO) is a full-featured testing suite that’s achieved tremendous popularity in the last two years. Overall, it’s very similar to Optmizely but with a few key differences.

At $49 per month for the starting price point, it’s not exactly the “Why not?” pricing of Optimizely, but you can expect a much wider feature set for the money. VWO boasts a simple, easy-to-use interface and comes with an impressive 7 different testing options:

  1. A/B Test
  2. Multivariate Test
  3. Split URL Test
  4. Targeting
  5. Usability Test
  6. Conversion Tracking
  7. Heatmap and Clicks

The software also includes a handful of  idea-generation tools. This is a unique selling point for business owners unsure of where to start in creating and running A/B tests.

Overall, VWO’s benefit to your company depends on your needs. If you need something dirt cheap to get started, or something very advanced, Optimizely is probably the way to go. If you’re needing beginner- to intermediate-level testing options at a traffic level greater than 1,000 visitors per month, VWO is the perfect tool for your business.

The Pros

  • Feature-rich for the price – full feature set available at all price points
  • Intuitive, easy-to-use interface
  • Idea-generation tools

The Cons

  • Lacks same level of real-time support options compared to competitors
  • Lacks multi-page testing options
  • Hit or miss online documentation

2nd Opinion

“Overall, Visual Website Optimizer has the slight edge for casual A/B testers, but its more expensive than the basic version of Optimizely. For serious A/B testers, I definitely recommend Optimizely – more comprehensive offering, even though you have to pay extra for premium features.”

Rich Page, Website Optmization Consultant

4. Unbounce

unbounce screen

Unbounce has carved out a niche for itself by focusing solely on landing pages. That’s it. Just landing pages. And they’ve managed to make themselves somewhat of an authority on the topic as a result.

Unbounce’s unique distinction is that they offer 80+ pre-designed, optimized landing page templates. This is a fantastic resource for business owners starting from scratch or who have seen little success with their own designs. Each template is engineered by the Unbounce team for the purpose of conversions and can be quickly customized for your company’s use.

In addition to templates, an Unbounce subscription will allow you to perform A/B testing on your landing page. You’ll be given a few metrics for interpreting the data, and that’s it. Other than a few periphery extras for the higher-priced options, the full Unbounce experience consists of landing page templates and A/B split testing.

If you’re a true marketing rookie and want results fast, this is probably your best bet. Few other programs cater to ground-zero quite like Unbounce, and you’ll be able to publish conversion-optimized landing pages in a matter of hours. To sum things up, Unbounce is limited in its goals but accomplishes those goals admirably.

The Pros

  • Landing page templates provide great start-from-scratch resource
  • Solid split testing capablities
  • Simple, easy-to-use interface
  • Designed to easily integrate with other marketing and testing software

The Cons

  • Limited to simple A/B split testing
  • Relatively high cost for a limited feature set

2nd Opinion

When creating a new landing page, Unbounce is our company’s go-to for A/B testing conversions. It is quick, gets the job done, and the analytics are actionable. We can’t live without its confidence score rating, which tells you how likely a good conversion rate is, or whether it is mere luck.

 –  Brian CurlissCEO MailLift

5. KISSmetrics

kissmetrics screen

Unless you’ve been living in a box for the last few years, you’ve heard of KISSmetrics. This popular testing tool focuses on the human side of data. Reports and testing data are connected to real people, who are tracked across their interactions with your website.

At $150 per month to start (with full-year commitment), KISSmetrics is definitely a premium service, but it comes with the advanced abilities you’d expect at that price point. You won’t find software offering deeper or more diverse sets of data. KISSmetrics lets you keep a pulse on the entire conversion funnel, from start to finish, and identify where and how customers are disconnecting from your website.

If you are serious about data, KISSmetrics is one of the best tools around. Its ability to connect data to actual users might also be a great alternative for business owners struggling to wrap their heads around more traditional reporting styles.

The Pros

  • Highly flexible data reporting options
  • Simple, intuitive interface
  • Can trace data to real people
  • Excels  at funnels with extremely accurate funnel-based data
  • Integrates well with complementary software

The Cons

  • Lower powered A/B testing compared to other testing software
  • High cost – won’t be affordable for many small businesses
  • More advanced data reporting requires a higher learning curve

2nd Opinion

“I use other split-testing tools, such as Optimizely, but KISSmetrics is a must if you’re serious about getting accurate results. The problem with many of the tools out there is they don’t measure the impact of the test throughout your entire funnel. That’s why I love KISSmetrics; it will show you exactly what the impact is for a test at every step of the buying process (not just the next step).”

Ruben Gamez, Founder Bidsketch

6. Crazy Egg

sample heatmap crazyegg

Of course, our list wouldn’t be complete without the home team’s offering. Crazy Egg specializes in click-tracking. The goal of all testing is to understand user behavior, and Crazy Egg’s software allows business owners to identify exactly what users are doing when they visit your website.

If you’re a visual learner, Crazy Egg will be your go-to source for data. The six different available reports frame data in an easily digestible way that is intuitive and provides immediate insight into user activity, regardless of your analytical skills.

If you’re at least capable of building a landing page, Crazy Egg is the ultimate entry-level option. At $9 per month, it is the cheapest testing software you’ll find anywhere.

While it is decidedly limited in scope and test variety, you’ll find it much easier to use than Google Analytics and most other options on this list. It is, perhaps, the only offering on this list that allows business owners to immediately read and understand test data with zero context for web analytics.

The Pros

  • Highly visual data – enjoyable to view
  • The benchmark of interface simplicity
  • Extremely low learning curve with intuitive reporting
  • Cheapest pricing you’ll find for testing software

The Cons

  • Limited feature sets and testing options
  • Limited utility for dynamic pages

2nd Opinion

“It’s a great tool considering how inexpensive it is. If used correctly and you have a plan for tracking your websites, this tool can make a world of a difference.”

Amber Steinert, Marketing Analyst 3M

Read other Crazy Egg articles by Jacob McMillen.



Get updates on new articles, webinars and other opportunities:

Jacob McMillen

Jacob McMillen is a website copywriter and content strategist. He helps businesses stop playing around with content marketing and start seeing tangible ROI. Download his free guide: 2 Fail-Proof Marketing Strategies For Businesses On A Budget


Comment Policy

Please join the conversation! We like long and thoughtful communication.
Abrupt comments and gibberish will not be approved. Please, only use your real name, not your business name or keywords. We rarely allow links in your comment.
Finally, please use your favorite personal social media profile for the website field.


Your email address will not be published.

  1. John says:
    June 17, 2015 at 8:57 am

    In order to take customers onboard your modification team and to minimize the chances of revenue and customer loss, A/B testing has become the sure shot way for various business organizations. A/B testing not only helps in smooth transition of the changes but also helps in improving the user experience and conversion rates of a website. Some of the tools that can help in this direction are Google Analytics, Unbounce, Optimizely and such others. I would also like to mention Mocking Fish as it is lower in cost, provides more accurate real time results and is easy to implement.

    • Kathryn Aragon says:
      June 17, 2015 at 2:06 pm

      Thanks, John. We’ll check Mocking Fish out.

  2. Irit Gillath says:
    April 29, 2015 at 3:29 pm

    Another tool that I like for heat maps and visits stats is

  3. sandybrown says:
    April 27, 2015 at 6:53 am

    google Analytics is best traffic analyser.

  4. Scotty Truong says:
    April 26, 2015 at 1:17 am

    Thanks for this detailed review. Would love to add 3MinuteOptimizer (disclaimer: I’m the founder of the business) which focus on exclusively on setting up split URL tests fast. Hope we could make it to the list.

    • Kathryn Aragon says:
      April 26, 2015 at 8:55 am

      Thanks for letting us know, Scotty.

  5. Peter says:
    April 23, 2015 at 8:39 am

    I would like to know if this article still holds good? or is it outdated as some of the A/B testing tools mentioned here may have worked on the cons?

  6. Pete says:
    January 23, 2015 at 7:08 am

    Brilliant, mission accomplished!!! We have researched kissmetrics and find that, although the functions are amazing, its not financially viable for us. Thanks.

    • Kathryn Aragon says:
      January 23, 2015 at 9:46 am

      Glad you arrived at a decision, Pete. Most businesses have a wish list of products. KISSmetrics may be a good for one down the road.

  7. sandybrown says:
    January 15, 2015 at 12:40 am

    software is save your time.

    • Kathryn Aragon says:
      January 15, 2015 at 9:15 am

      Agreed, Sandybrown. The best testing software DOES save you time.

    • Neil Patel says:
      January 15, 2015 at 12:01 pm

      Sandy, it definitely does!

  8. Andrei says:
    January 13, 2015 at 5:15 am

    You also forget such tool as a
    It’s the easiest A/B testing toll based on Google Experiments API, so it’s fully integrated with your account in Google Analytics. So all the data is gathered with maximum accuracy.

    • Kathryn Aragon says:
      January 13, 2015 at 9:22 am

      Thanks, Andrei. Appreciate the addition.

    • Neil Patel says:
      January 13, 2015 at 12:06 pm

      Andrei, thanks for adding that one and providing the additional feedback.

  9. Chris says:
    January 9, 2015 at 1:06 pm

    How is Adobe’s Test & Target tool not on this list? In the testing community it is arguably THE a/b testing tool.

    • Kathryn Aragon says:
      January 9, 2015 at 2:15 pm

      Thanks for the addition, Chris. What’s the cost? I couldn’t find it.

  10. June 26, 2014 at 10:18 am

    Hi Jacob,

    Hope makes it to the next list.


  11. Tia Kelly says:
    June 26, 2014 at 1:53 am

    Hey Jacob! Really appreciate you mentioning Unbounce in this list. Have to give props to the other 5 you chose to highlight here 🙂 All do a great job in their specialty. I also liked the “second opinion” quotes—something different that I haven’t really seen done before in a top tools type of post that adds credibility. Thanks!

    Social Strategist, Unbounce

    • June 26, 2014 at 2:26 pm

      Thanks Tia. My goal is that it will make choosing software easier for business owners.

      • Neil Patel says:
        June 26, 2014 at 5:59 pm

        Jacob, Im sure it will. Thanks for the feedback 🙂

  12. Zac Aghion says:
    June 25, 2014 at 2:40 pm

    Nice review Jacob, thanks!

    Google Content Experiments can also be used to run automated A/B tests. From a high level, the way this approach works is to start showing variations that work better more often, automatically. It’s a much better solution to the question of ‘What works better: A or B?”

    In the most basic example of an A/B test, you have a variation A and a variation B each shown to 50% of your user base. By definition, this approach will be sending half of your users to a worse performing version during the entire duration of the test!

    The automated approach is based on a bandit algorithm that updates the percentage of your user base which is shown a given variation dynamically. With each new piece of data that you collect on the conversion rate and statistical confidence of the variations in a test, it adjusts the percentages automatically so that better performing variations are promoted and worse performers are pruned away.

    This leads to:

    1) faster results, because your directing test resources (i.e.: users and their data) to validate what you actually care about (i.e.: confidence in the best variation’s performance)

    2) a higher average conversion rate during the test itself, because relatively more users are being sent to the better performing variation automatically, and

    3) less time and effort required to actively manage your experiments.

    Though the math behind this approach is slightly more complex than a traditional A/B test, it’s worth talking about for audiences that are interested in making data-driven decisions because of how much better the results are that it produces.

    For anyone interested, here’s a post we put together on how it works:

    • June 25, 2014 at 2:52 pm

      Great insights, Zac. Thanks for sharing your expertise.

    • Brian Lang says:
      June 26, 2014 at 10:36 am

      Great comment Zac – I’m convinced minimizing regret is the way to go, but I think there are limitations/concerns with existing offerings – Auto-optimization sounds great in theory, but the reality is it isn’t as simple.

      For example, most of the available tools haven’t implemented contextually aware auto-optimization methods, so you ultimately end up creating a lot more work implementing the same experiment for many different user segments (a time consuming pain point), or you settle for, “what’s the best option for most”.

      Separately, overlapping experiments with an auto-optimization setup, where one experiments better-performing arm can inadvertently skew (positively) the results of another experiments inferior arm, is a real problem, and most tools available don’t offer a solution to it.

      There are also issues with traffic quality and returning customer bias (through things like the recency/novelty effect) in-advertently skewing results of one arm, when it in fact might be the inferior one – in this situation, you would in-advertently be maximizing, not minimizing regret, and depending on the method of your “stopping point criteria”, might lead to wrong conclusions being made.

      A frequentist approach *typically* doesn’t suffer the above issues, since you are typically doing a 50/50 split of traffic, and can then segment results. There are, of course, ways to address these issues in an auto-optimization setup, but as of yet I haven’t really found a tool that does so.

      Thanks again!


Show Me My Heatmap

To capture data, @blueearth recommends website heatmaps #CrushinItMN



What makes people leave your website?