{"id":43308,"date":"2025-07-02T07:00:00","date_gmt":"2025-07-02T11:00:00","guid":{"rendered":"https:\/\/crazyeggblog.wpengine.com\/?p=43308"},"modified":"2025-07-14T16:39:30","modified_gmt":"2025-07-14T20:39:30","slug":"multivariate-testing","status":"publish","type":"post","link":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/","title":{"rendered":"What Multivariate Testing Solves That AB Testing Can\u2019t"},"content":{"rendered":"\n<p>Multivariate testing isn\u2019t better or worse than AB testing. They solve different problems.<\/p>\n\n\n\n<p>You can run 90+ percent of the tests you will ever need on a website with <a href=\"https:\/\/www.crazyegg.com\/ab-testing\" target=\"_blank\" rel=\"noreferrer noopener\">A\/B testing<\/a>. But there are a few key questions it cannot answer.<\/p>\n\n\n\n<p>Can multivariate testing help you figure out the best next move? Here\u2019s what you need to know.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What Is Multivariate Testing?<\/h2>\n\n\n\n<p>Multivariate testing (MVT) is a controlled experiment which allows people to examine the impact of changing two or more page elements on a website at the same time.<\/p>\n\n\n\n<p>The core objective of multivariate testing is almost always to find a combination of page elements that has a higher <a href=\"https:\/\/www.crazyegg.com\/blog\/what-is-good-conversion-rate\/\" target=\"_blank\" rel=\"noreferrer noopener\">conversion rate<\/a> than the existing page. The conversion companies care about is usually sales, signups, or some other revenue-driving metric.<\/p>\n\n\n\n<p>Using MVT, brands can answer questions like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>What is the highest-converting combination of elements?<\/li>\n\n\n\n<li>Which elements work well together?<\/li>\n\n\n\n<li>Which elements have the most\/least influence on conversions?<\/li>\n<\/ul>\n\n\n\n<p>The type of multivariate testing used on websites has its roots in experimental designs from behavioral sciences and industrial manufacturing. It\u2019s helped manufacturers perfect intricate assembly lines, and allowed pharmaceutical companies to isolate drug interactions inside the human body.<\/p>\n\n\n\n<p>MVT is perfect for websites, which have diverse and fluctuating traffic. There are so many variables that impact whether a user converts.<\/p>\n\n\n\n<p>Multivariate testing can help you start investigating how different layouts perform and how different elements interact with one another. You\u2019ll discover what really matters, and be able to predict the best performing possible combination of page elements.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How Multivariate Testing Works<\/h2>\n\n\n\n<p>MVT testing starts with a high-value, high-traffic page that you think can convert better than it is right now.<\/p>\n\n\n\n<p>Once you and your team know which elements on this page you want to test, you start building out the different page experiences.&nbsp;<\/p>\n\n\n\n<p>This is almost always done in a specialized testing platform, which makes it easy to create different variations of page elements, track conversions, and split traffic between the different page experiences you are testing.<\/p>\n\n\n\n<p>In the simplest possible test, where you make a single change to 2 different elements, there will be combinations. For example:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Headline A, Image A (control)<\/li>\n\n\n\n<li>Headline A, Image B<\/li>\n\n\n\n<li>Headline B, Image A<\/li>\n\n\n\n<li>Headline B, Image B<\/li>\n<\/ol>\n\n\n\n<p>Each of these different combinations provides a slightly different <a href=\"https:\/\/www.crazyegg.com\/blog\/what-is-user-experience\/\" target=\"_blank\" rel=\"noreferrer noopener\">user experience (UX)<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1200\" height=\"720\" src=\"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png\" alt=\"Graphic to show how multivariate testing works with four different phone screens displayed with different views. \" class=\"wp-image-104068\" srcset=\"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png 1200w, https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration-300x180.png 300w, https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration-1024x614.png 1024w, https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration-768x461.png 768w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<p>Typically, traffic is randomized and then divided evenly between all possible combinations of page elements. In the above example, each page experience would receive 25% of the traffic.<\/p>\n\n\n\n<p>After the test is complete, you will be able to see the performance metrics of all the different page experiences, measured against the control.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">MVT vs AB testing<\/h2>\n\n\n\n<p><a href=\"https:\/\/www.crazyegg.com\/blog\/ab-testing\/\" target=\"_blank\" rel=\"noreferrer noopener\">AB testing<\/a> is constrained to changing a single page element. This is a feature, not a bug.<\/p>\n\n\n\n<p>Because you are only changing one element, you can get really great data on that specific element. While MVT tells you about the effect of elements in concert, AB testing isolates individual elements to see their direct effect.<\/p>\n\n\n\n<p>AB testing is phenomenal for validating your creative and strategic assumptions about what users want on a webpage.<\/p>\n\n\n\n<p>But AB testing can only test two different experiences at once.<\/p>\n\n\n\n<p>Even if you create an AB test where you change a whole bunch of elements on one page, you are still only testing two different experiences.<\/p>\n\n\n\n<p>In order to test multiple experiences with AB testing, you have to run a series of AB tests, proceeding sequentially through different combinations of page elements.<\/p>\n\n\n\n<p>Companies use sequential AB testing, and it is valuable. It\u2019s just slow, running one test after another. With MVT, you can run through many of those different combinations of elements at the same time.<\/p>\n\n\n\n<p>That\u2019s one of the big selling points. Teams can iterate really fast because they are assessing so many different combinations at once.<\/p>\n\n\n\n<p>They also get to see how different <a href=\"https:\/\/www.crazyegg.com\/blog\/persuasion-techniques\/\" target=\"_blank\" rel=\"noreferrer noopener\">persuasion techniques<\/a> and <a href=\"https:\/\/www.crazyegg.com\/blog\/typography-elements\/\" target=\"_blank\" rel=\"noreferrer noopener\">typography elements<\/a> interact, and what effect this has on conversions.<\/p>\n\n\n\n<p>But as I said to start this post, AB testing and MVT testing are both useful tools with different jobs.<\/p>\n\n\n\n<p>AB testing is less expensive, easier to set up, requires way less traffic, and in many cases, it provides more valuable business insights. Which elements matter at all? What messaging and images resonate with users?<\/p>\n\n\n\n<p>Most businesses that use MVT have already AB tested heavily to figure out a design they want to further refine.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why Does MVT Need So Much Traffic?<\/h2>\n\n\n\n<p>You may have heard this, and it\u2019s true: multivariate testing requires a ton of traffic.<\/p>\n\n\n\n<p>There are two contributing factors.<\/p>\n\n\n\n<p>First, you have to get a lot of people through a test in order to know that the results aren\u2019t random. Just think, you can\u2019t be as confident about a test with 10 participants as you can about a test with 1,000 participants.<\/p>\n\n\n\n<p>Second, you need to get enough traffic to every experience you want to test. In AB testing, you usually split traffic in half, 50% goes to the control 50% goes to the variation.<\/p>\n\n\n\n<p>In the least complex possible MVT design, you are splitting traffic between four experiences, with each getting 25% of your website traffic.<\/p>\n\n\n\n<p>The more elements you change, the higher the number of possible combinations, and the more your traffic is divided. For example:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>3 headlines + 2 images = 6 total experiences<\/li>\n\n\n\n<li>3 headlines + 3 images + 2 CTA styles = 18 total experiences<\/li>\n\n\n\n<li>4 headlines + 3 images + 3 CTA styles = 36 total experiences<\/li>\n<\/ul>\n\n\n\n<p>There are very few websites in the world that could supply enough traffic to 36 different experiences simultaneously.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Full-factorial vs fractional MVT<\/h3>\n\n\n\n<p>When you run all possible combinations of experiences, it is known as <strong>full-factorial MVT<\/strong>.<\/p>\n\n\n\n<p>But you can also elect to run <strong>fractional MVT<\/strong>, which is where not all possible combinations are tested. Test designers can pick and choose a fraction of the total possible combinations to test.<\/p>\n\n\n\n<p>There is an obvious risk here of not getting complete data, but it decreases the traffic requirements significantly.&nbsp;<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Estimating MVT traffic requirements<\/h3>\n\n\n\n<p>Many websites don\u2019t have enough traffic to split their traffic 50\/50 for AB testing, let alone split it 4+ ways to run MVT.<\/p>\n\n\n\n<p>For AB testing, the rule of thumb is 8,000 web page visitors per month, which should get 1,000 visitors through both experiences in a week. This is usually what is required to hit statistical significance within a reasonable timeframe under normal conditions.<\/p>\n\n\n\n<p>For MVT, even the least complex design (4 experiences) requires 16,000 visitors per month to get 1,000 users through each experience.&nbsp;<\/p>\n\n\n\n<p>If you start testing lots of page elements and including multiple variations, the required traffic quickly jumps into the 100k+ visitors per month.<\/p>\n\n\n\n<p><a href=\"https:\/\/www.crazyegg.com\/blog\/increase-website-traffic\/\" target=\"_blank\" rel=\"noreferrer noopener\">Driving more traffic to your site<\/a> is always a good idea, but it takes time. If you are interested in digging into individual page performance, but you are still a ways out from the traffic requirements of&nbsp; MVT or AB testing, consider using <a href=\"https:\/\/www.crazyegg.com\/blog\/website-heatmap\/\" target=\"_blank\" rel=\"noreferrer noopener\">website heatmaps<\/a>, click maps, and session recordings. These tools can provide critical insights without needing much traffic at all.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Common Page Elements To Test With MVT<\/h2>\n\n\n\n<p>Because MVT is expensive conduct, businesses only pursue MVT on web pages that drive significant revenue, and only spend time testing page elements that are likely to impact the <a href=\"https:\/\/www.crazyegg.com\/blog\/what-is-good-conversion-rate\/\" target=\"_blank\" rel=\"noreferrer noopener\">conversion rate<\/a>.&nbsp;<\/p>\n\n\n\n<p>These are some of the typical page elements targeted for MVT testing:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.crazyegg.com\/blog\/what-is-a-headline\/\" target=\"_blank\" rel=\"noreferrer noopener\">Headlines<\/a><\/li>\n\n\n\n<li>Featured images<\/li>\n\n\n\n<li>Call-out boxes<\/li>\n\n\n\n<li>Special offer language<\/li>\n\n\n\n<li>Form field length<\/li>\n\n\n\n<li>Form field labels<\/li>\n\n\n\n<li><a href=\"https:\/\/www.crazyegg.com\/blog\/social-proof\/\" target=\"_blank\" rel=\"noreferrer noopener\">Social proof<\/a><\/li>\n\n\n\n<li>Trust signals<\/li>\n\n\n\n<li><a href=\"https:\/\/www.crazyegg.com\/blog\/call-to-action-examples\/\" target=\"_blank\" rel=\"noreferrer noopener\">Calls to action (CTAs)<\/a><\/li>\n<\/ul>\n\n\n\n<p>There are other potential page elements to experiment with, but you want to be as sure as possible that you are only testing elements that will actually influence user behavior.<\/p>\n\n\n\n<p>Brands will also use MVT to analyze the success of different layouts, often moving these target elements higher on the page to see which ones lead to improvements.<\/p>\n\n\n\n<p>One easy way to find impactful page elements is to run a <a href=\"https:\/\/www.crazyegg.com\/blog\/conversion-audit\/\" target=\"_blank\" rel=\"noreferrer noopener\">quick CRO audit<\/a> on the page to determine if there are obvious places on the site where you are losing conversions. Focusing changes on elements that can address those issues is likely to have a meaningful effect.<\/p>\n\n\n\n<p>If you are truly at the refining stage with MVT testing, you\u2019ll have to get more creative. I would find inspiration from your competitors\u2019 websites. What are they doing to convince buyers in your space?<\/p>\n\n\n\n<p>You can also look at <a href=\"https:\/\/www.crazyegg.com\/blog\/landing-page-essentials\/\" target=\"_blank\" rel=\"noreferrer noopener\">landing pages<\/a> of brands who are buying sponsored ad space in Google search results. The companies spending lots of money on ads are testing their landing pages heavily. You may be able to snipe a few new tricks from them.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How To Run Multivariate Testing<\/h2>\n\n\n\n<p>Everyone I know uses some sort of testing platform to run MVT on websites. These tools enforce traffic randomization, automate the calculations, and provide an easy interface to create and analyze tests.<\/p>\n\n\n\n<p>The testing tool you use will shape the specific workflow, but virtually every MVT method follows a similar process:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Select a page<\/strong>: It must have lots of traffic and (in most cases) you should be happy with the overall design as MVT works best for refining specific elements.<\/li>\n\n\n\n<li><strong>Select goal<\/strong>: This is how the performance of each experience is assessed. The goal must be a trackable metric (like clicks on a specific element) that\u2019s tied to a meaningful business objective.<\/li>\n\n\n\n<li><strong>Create experiences<\/strong>: This involves selecting elements and creating variations in the MVT testing platform editor. Every variation results in more potential combinations of website elements that users could experience.<\/li>\n\n\n\n<li><strong>Estimate the existing performance<\/strong>: This is the baseline, normal rate of the page that you are testing. During the test you will use the actual performance of the existing page to measure, but this estimation is important for estimating an appropriate test duration.<\/li>\n\n\n\n<li><strong>Confirm statistical power and significance level<\/strong>: Most testing platforms default to 80% statistical power and 95% significance level. Only adjust these if you have a good reason.<\/li>\n\n\n\n<li><strong>Set improvement threshold<\/strong>: This is the change you are looking to detect. If your goal is clicks on a specific element, for example, do you want to see a 10% lift to the conversion rate? A 20% lift? Smaller improvements need a larger sample size to detect with confidence compared to large improvements.<\/li>\n\n\n\n<li><strong>Estimate test duration<\/strong>: This is automatically calculated based on your goals, improvement threshold, the total number of combinations, and the average number of monthly visitors that come to your site. The lower the traffic, the longer the test will have to run.<\/li>\n\n\n\n<li><strong>Launch the test<\/strong>: Let it run and don\u2019t peek at the results. Interfering with the test jeopardizes the validity, which means you can\u2019t be certain any results are repeatable.<\/li>\n<\/ol>\n\n\n\n<p>The testing platform will alert you when enough users have experienced all of the combinations to have achieved statistical significance. This means that any effect you detected with the test is not due to random chance.<\/p>\n\n\n\n<p>You can safely stop the test at this point. 100% of your website traffic will return to your existing URL.<\/p>\n\n\n\n<p>Now let&#8217;s take a closer look at some of the key concepts in the MVT process.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Statistical significance in multivariate testing<\/h3>\n\n\n\n<p>There is a lot of randomness on a website. Users could be on their phones, at a desk, talking to their children, making dinner, or fighting with a lousy internet connection.<\/p>\n\n\n\n<p>Based on circumstances entirely outside of your control, the same user could have a completely different experience on your site from one day to the next.<\/p>\n\n\n\n<p>Both AB tests and multivariate tests must run for a <strong>long enough duration<\/strong> and test a <strong>large enough sample size<\/strong> of users to achieve statistical significance.&nbsp;<\/p>\n\n\n\n<p>This is a key concept in <a href=\"https:\/\/www.crazyegg.com\/blog\/hypothesis-testing\/\" target=\"_blank\" rel=\"noreferrer noopener\">hypothesis testing<\/a>, and tells researchers that they have enough data to conclude that the outcome was not just due to random chance.<\/p>\n\n\n\n<p>If you stop your test before it has reached statistical significance, you cannot be confident in the results.&nbsp;<\/p>\n\n\n\n<p>Even if you saw an amazing conversion lift. Doesn\u2019t matter. The change you observed might just be due to random chance.<\/p>\n\n\n\n<p>An easy way to think about statistical significance is to imagine a survey that 10 people have responded to. Would you take those results as seriously as a survey that has 1,000 responses?<\/p>\n\n\n\n<p>Of course not.&nbsp;<\/p>\n\n\n\n<p>The larger sample size gives us more common sense confidence that the survey results reflect what we\u2019d see again if we asked 1,000 more people.<\/p>\n\n\n\n<p>So how long do you have to let the test run? How many users need to go through the test?<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Calculating MVT sample size and duration<\/h3>\n\n\n\n<p>The testing platform will do all of this for you with a few basic inputs about the page you are testing. These are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Average monthly traffic<\/strong>: The number of visitors who land on the page, typically<\/li>\n\n\n\n<li><strong>Existing conversion rate<\/strong>: the current baseline, normal average.<\/li>\n\n\n\n<li><strong>Improvement threshold<\/strong>: The minimum change in the conversion rate you want to detect.<\/li>\n\n\n\n<li><strong>Statistical power<\/strong>: Usually set to 80%, this is the likelihood your test will detect a real effect, if one exists.<\/li>\n\n\n\n<li><strong>Significance level<\/strong>: Usually set to 95%, this is how confident you want to be that your results aren&#8217;t due to chance.<\/li>\n\n\n\n<li><strong>Total number of experiences<\/strong>: The total combinations being tested, based on the number of variables and variations.<\/li>\n<\/ul>\n\n\n\n<p>Once you plug all the information in, the platform will calculate how many weeks the test needs to run.<\/p>\n\n\n\n<p>Ideally it will say something close to 2 weeks. That\u2019s a good amount of time to run an MVT test. A week can work, but there can be a lot of randomness week to week that your test won\u2019t be able to pick up on.<\/p>\n\n\n\n<p>If the duration is calculated out at like 10-20+ weeks, consider reducing the number of variations and testing higher-impact elements only. This will split your traffic among fewer total experiences, allowing more users to get through the test quicker.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Analyzing MVT Test Results<\/h2>\n\n\n\n<p>The key results that for each combination are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Conversion rate<\/strong>: Percentage of conversions to total traffic. Platforms may provide the rate as a fixed number or range.<\/li>\n\n\n\n<li><strong>Improvement<\/strong>: Percentage of change compared to the control. Positive percentage for improvement, negative percentage for decline.<\/li>\n\n\n\n<li><strong>Confidence level<\/strong>: Percentage expressing how likely the changes you made are responsible for the outcomes you see.<\/li>\n<\/ul>\n\n\n\n<p>A clear winner would have a large improvement with a high confidence level.<\/p>\n\n\n\n<p>For MVT on websites, most people are shooting for 95% confidence level, which basically means there is only a 5% chance that the outcome was random.<\/p>\n\n\n\n<p>So if you saw an improvement of 20% with a 96% percent confidence level, that would be fantastic.<\/p>\n\n\n\n<p>But it\u2019s not always so cut and dry. Sometimes there is not a clear winner, much improvement, or a high enough confidence level about the result.<\/p>\n\n\n\n<p>In these cases, you can still sometimes find wins with particular traffic segments. Most testing tools allow you to look at traffic from different times of day, countries, referrers, and so on.<\/p>\n\n\n\n<p>Where there isn\u2019t a general trend that stands out, sometimes you can find a win with a particular segment of your <a href=\"https:\/\/www.crazyegg.com\/blog\/target-audience\/\" target=\"_blank\" rel=\"noreferrer noopener\">target audience<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Multivariate testing isn\u2019t better or worse than AB testing. They solve different problems. You can run 90+ percent of the tests you will ever need&#8230;<\/p>\n","protected":false},"author":254,"featured_media":104068,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_eb_attr":"","_lmt_disableupdate":"","_lmt_disable":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[4],"tags":[],"class_list":["post-43308","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-conversion-optimization"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>What Multivariate Testing Solves That AB Testing Can\u2019t<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What Multivariate Testing Solves That AB Testing Can\u2019t\" \/>\n<meta property=\"og:description\" content=\"Multivariate testing isn\u2019t better or worse than AB testing. They solve different problems. You can run 90+ percent of the tests you will ever need...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"The Daily Egg\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/crazyegganalytics\/\" \/>\n<meta property=\"article:published_time\" content=\"2025-07-02T11:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-14T20:39:30+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"720\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Peter Lowe\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@CrazyEgg\" \/>\n<meta name=\"twitter:site\" content=\"@CrazyEgg\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Peter Lowe\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"17 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/\"},\"author\":{\"name\":\"Peter Lowe\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#\\\/schema\\\/person\\\/526c464e0c3c4513698e8b5823e00989\"},\"headline\":\"What Multivariate Testing Solves That AB Testing Can\u2019t\",\"datePublished\":\"2025-07-02T11:00:00+00:00\",\"dateModified\":\"2025-07-14T20:39:30+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/\"},\"wordCount\":2496,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ceblog.s3.amazonaws.com\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/02150820\\\/Multivariate-Testing-Illustration.png\",\"articleSection\":[\"Conversion\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/\",\"url\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/\",\"name\":\"What Multivariate Testing Solves That AB Testing Can\u2019t\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/ceblog.s3.amazonaws.com\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/02150820\\\/Multivariate-Testing-Illustration.png\",\"datePublished\":\"2025-07-02T11:00:00+00:00\",\"dateModified\":\"2025-07-14T20:39:30+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#primaryimage\",\"url\":\"https:\\\/\\\/ceblog.s3.amazonaws.com\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/02150820\\\/Multivariate-Testing-Illustration.png\",\"contentUrl\":\"https:\\\/\\\/ceblog.s3.amazonaws.com\\\/wp-content\\\/uploads\\\/2019\\\/01\\\/02150820\\\/Multivariate-Testing-Illustration.png\",\"width\":1200,\"height\":720,\"caption\":\"Graphic to show how multivariate testing works with four different phone screens displayed with different views.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/multivariate-testing\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Blog\",\"item\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Conversion\",\"item\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/category\\\/conversion-optimization\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"What Multivariate Testing Solves That AB Testing Can\u2019t\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/\",\"name\":\"The Daily Egg\",\"description\":\"Conversion Rate Optimization Made Easy\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#organization\",\"name\":\"Crazy Egg\",\"url\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/ceblog.s3.amazonaws.com\\\/wp-content\\\/uploads\\\/2015\\\/06\\\/Crazy-Egg-logo-small.png\",\"contentUrl\":\"https:\\\/\\\/ceblog.s3.amazonaws.com\\\/wp-content\\\/uploads\\\/2015\\\/06\\\/Crazy-Egg-logo-small.png\",\"width\":191,\"height\":100,\"caption\":\"Crazy Egg\"},\"image\":{\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/crazyegganalytics\\\/\",\"https:\\\/\\\/x.com\\\/CrazyEgg\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/crazy-egg\\\/\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCJNe_xmPi07YezxaqfoRVqg\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/#\\\/schema\\\/person\\\/526c464e0c3c4513698e8b5823e00989\",\"name\":\"Peter Lowe\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ee86f41fc6f31ebf4977c622dc46568df1d0ae58dd64405ab97403cb37a04c4e?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ee86f41fc6f31ebf4977c622dc46568df1d0ae58dd64405ab97403cb37a04c4e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ee86f41fc6f31ebf4977c622dc46568df1d0ae58dd64405ab97403cb37a04c4e?s=96&d=mm&r=g\",\"caption\":\"Peter Lowe\"},\"description\":\"Peter works in marketing and lead generation. He has been involved on the backend of the Crazy Egg blog since 2020. Occasionally, they let him write a post.\",\"url\":\"https:\\\/\\\/www.crazyegg.com\\\/blog\\\/author\\\/peter\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What Multivariate Testing Solves That AB Testing Can\u2019t","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/","og_locale":"en_US","og_type":"article","og_title":"What Multivariate Testing Solves That AB Testing Can\u2019t","og_description":"Multivariate testing isn\u2019t better or worse than AB testing. They solve different problems. You can run 90+ percent of the tests you will ever need...","og_url":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/","og_site_name":"The Daily Egg","article_publisher":"https:\/\/www.facebook.com\/crazyegganalytics\/","article_published_time":"2025-07-02T11:00:00+00:00","article_modified_time":"2025-07-14T20:39:30+00:00","og_image":[{"width":1200,"height":720,"url":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png","type":"image\/png"}],"author":"Peter Lowe","twitter_card":"summary_large_image","twitter_creator":"@CrazyEgg","twitter_site":"@CrazyEgg","twitter_misc":{"Written by":"Peter Lowe","Est. reading time":"17 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#article","isPartOf":{"@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/"},"author":{"name":"Peter Lowe","@id":"https:\/\/www.crazyegg.com\/blog\/#\/schema\/person\/526c464e0c3c4513698e8b5823e00989"},"headline":"What Multivariate Testing Solves That AB Testing Can\u2019t","datePublished":"2025-07-02T11:00:00+00:00","dateModified":"2025-07-14T20:39:30+00:00","mainEntityOfPage":{"@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/"},"wordCount":2496,"commentCount":0,"publisher":{"@id":"https:\/\/www.crazyegg.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png","articleSection":["Conversion"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/","url":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/","name":"What Multivariate Testing Solves That AB Testing Can\u2019t","isPartOf":{"@id":"https:\/\/www.crazyegg.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#primaryimage"},"image":{"@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#primaryimage"},"thumbnailUrl":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png","datePublished":"2025-07-02T11:00:00+00:00","dateModified":"2025-07-14T20:39:30+00:00","breadcrumb":{"@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#primaryimage","url":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png","contentUrl":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2019\/01\/02150820\/Multivariate-Testing-Illustration.png","width":1200,"height":720,"caption":"Graphic to show how multivariate testing works with four different phone screens displayed with different views."},{"@type":"BreadcrumbList","@id":"https:\/\/www.crazyegg.com\/blog\/multivariate-testing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Blog","item":"https:\/\/www.crazyegg.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Conversion","item":"https:\/\/www.crazyegg.com\/blog\/category\/conversion-optimization\/"},{"@type":"ListItem","position":3,"name":"What Multivariate Testing Solves That AB Testing Can\u2019t"}]},{"@type":"WebSite","@id":"https:\/\/www.crazyegg.com\/blog\/#website","url":"https:\/\/www.crazyegg.com\/blog\/","name":"The Daily Egg","description":"Conversion Rate Optimization Made Easy","publisher":{"@id":"https:\/\/www.crazyegg.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.crazyegg.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.crazyegg.com\/blog\/#organization","name":"Crazy Egg","url":"https:\/\/www.crazyegg.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.crazyegg.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2015\/06\/Crazy-Egg-logo-small.png","contentUrl":"https:\/\/ceblog.s3.amazonaws.com\/wp-content\/uploads\/2015\/06\/Crazy-Egg-logo-small.png","width":191,"height":100,"caption":"Crazy Egg"},"image":{"@id":"https:\/\/www.crazyegg.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/crazyegganalytics\/","https:\/\/x.com\/CrazyEgg","https:\/\/www.linkedin.com\/company\/crazy-egg\/","https:\/\/www.youtube.com\/channel\/UCJNe_xmPi07YezxaqfoRVqg"]},{"@type":"Person","@id":"https:\/\/www.crazyegg.com\/blog\/#\/schema\/person\/526c464e0c3c4513698e8b5823e00989","name":"Peter Lowe","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/ee86f41fc6f31ebf4977c622dc46568df1d0ae58dd64405ab97403cb37a04c4e?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/ee86f41fc6f31ebf4977c622dc46568df1d0ae58dd64405ab97403cb37a04c4e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ee86f41fc6f31ebf4977c622dc46568df1d0ae58dd64405ab97403cb37a04c4e?s=96&d=mm&r=g","caption":"Peter Lowe"},"description":"Peter works in marketing and lead generation. He has been involved on the backend of the Crazy Egg blog since 2020. Occasionally, they let him write a post.","url":"https:\/\/www.crazyegg.com\/blog\/author\/peter\/"}]}},"modified_by":"Lars Lofgren","_links":{"self":[{"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/posts\/43308","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/users\/254"}],"replies":[{"embeddable":true,"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/comments?post=43308"}],"version-history":[{"count":0,"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/posts\/43308\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/media\/104068"}],"wp:attachment":[{"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/media?parent=43308"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/categories?post=43308"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.crazyegg.com\/blog\/wp-json\/wp\/v2\/tags?post=43308"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}