In this post I am going to show you why qualitative analysis with surveys is important and how you can use FourEyes to get the insights you need. Let’s start with a case study to show you exactly what I mean.
If you’re a CRO geek, you probably remember this case study, where Conversion Rate Experts used long-form copy design strategies to help Crazy Egg achieve a 30% lift in conversions. This image may jog your memory:
Following up that performance was a tall order, but Joanna was undeterred. Here’s what she did to take Crazy Egg’s landing page to the next level.
(Hint: She combined heatmaps and surveys.)
(Double hint: That’s what this post will teach you to do, too.)
#amreading: How to Combine Heatmaps and Surveys to Boost Your CRO Wins @rajivkhaneja @CrazyEggClick to tweet
Joanna started by assessing Crazy Egg’s new long-form page, asking questions that only a mix of quantitative and qualitative research could answer:
- Was the long-form copy working throughout the page? Were there any weak spots?
- Which messages were essential to conversion… and which weren’t?
- What was better: leading with a product benefit or feature?
- Were the elements that come with long-form copy, like the Johnson Box, actually providing value and optimized for conversions?
To tackle the first question, Crazy Egg’s scroll-mapping tool was used to analyze Crazy Egg’s landing page. (How meta.)
The scroll-mapping tool showed the new long-form page was one tough mother to get through for visitors, starting red hot at the top of the page and then slowly but surely getting colder until visitors got to what they were looking for: testimonials and pricing.
As you can see, visitors’ mice were getting a workout once they began reading about the history of eye-tracking software and only started to slow when readers got to Crazy Egg’s testimonials and product pricing.
For long-form landing pages, scroll mapping is incredibly useful for identifying which sections are being passed over and which parts are attracting eyeballs. With this info visualized, you can edit copy or rearrange elements to improve the attention time on your landing page.
The other crazy awesome feature of Crazy Egg’s heat maps is that they’ll show you elements of your page that aren’t clickable, but are getting getting attention and clicks because your landing page visitors think they’re clickable. Crazy Egg helps you find these elements, so you can add a link to that valuable screen real estate.
With Crazy Egg’s heat map analysis in the books, it was time to add the second weapon for this CRO masterclass — surveys.
Survey Says? Clean Up The Copy
To start, Hiten had the bright idea to sort customers by the plan they were on to see if the groups responded differently to the survey questions.
Consider this when you make your own surveys — the more granular and specific you can get before sending out the survey, the better insights you’ll be able to glean from the results.
Again, because Joanna wasn’t trying to reinvent the wheel, the survey included questions focused on improving the copy of the page. The responses would then enhance the insights gathered from the heat map analysis.
Here were some of the exact questions used in the survey if you want to swipe them and repurpose them for your own surveys:
- What was going on in your world that caused you to seek out a solution like Crazy Egg?
- Give 1 example of something unexpected Crazy Egg has shown you that’s helped you optimize your site.
- Which of the following best describes your current role or title?
From these types of questions, the answers you get will help you write copy for the features or benefits of your product. You should be looking for the exact words they use and using those in your copy. In combo with quantitative data like that seen in heatmaps, qualitative survey data is your friend.
These survey answers, along with the heatmaps, formed a tag team that showed what elements of the page were getting the most clicks, plus the messaging that visitors would respond to that would ensure a significant conversion boost. To test all of their ideas for optimizing the page, four variations were created, as you can see below:
The best performing variation was “D” — the one all the way to the right, which brought a lift of 13%. Not jaw-dropping, sure, but pretty darn good! I’m sure you wouldn’t turn up your nose at 13% more signups, would you?
Here are just a few of the copy changes taken directly from survey insights:
- Would you believe that the best performing headline was pulled directly from phrasing used by a few survey respondents?
- And the three subheads were all lifted from answers to this survey question: What was going on in your world that compelled you to look for a solution like Crazy Egg?
- Segmenting between the plans proved invaluable: The site visitors likely to pay for the higher tier plans weren’t startups as originally suspected, but digital marketers, web managers and designers. This was added to the copy to show the job demographic Crazy Egg fits best.
This, in addition to the CRO boost from Crazy Egg’s page received from the Conversion Rate Experts, shows how effective CRO can be when you treat it like an iterative process.
By combining insights gleaned from tools like heatmaps and surveys, you can find hidden insights that lead to more conversions and profits!
Did You Say Profits? How Does $1 Million Sound?
That’s how much revenue Conversion Rate Experts made for Moz when they redesigned their landing page, creating a 52% boost in sales. How’d they do it? From insights gained from customer surveys, of course.
Here are some of the survey questions they asked paid members:
- What do you most like about our service?
- What convinced you to sign up?
- How would you describe the service to a friend?
Here are some survey questions they asked free-trial members:
- What would make you sign up for the paid version of Moz?
- What tools did you like the most (and the least)?
- What are your most time-consuming SEO tasks.
They also surveyed former paying members and asked them why they canceled. (Always be segmenting.) It was the insights from these questions that enabled CRE to get a 52% increase in a test sample of more than 5,000 site visitors.
How did Moz get such a large sample, you ask? After creating new wireframes based on the insights from their customer surveys, they asked their Twitter followers to participate in some user testing.
With such amazing results, Moz wasted no time going after their free-trial members, offering them a full membership for a month for just $1. A no-brainer, right?
Turns out their users are more skeptical than they thought, and Moz received over 500 emails all asking what the catch was with the offer. In response, Moz CEO Rand Fishkin sent out an email reassuring all their free-trial subscribers that there was no catch:
With the increase in conversion to paid memberships, plus the free-trial members who ended up converting after taking advantage of their literally unbelievable offer, Moz generated $1 million dollars to their annual revenue. And it all started with the insights they tested from their survey questions.
For example, survey results showed that many of Moz’s customers didn’t know all the features that were included in a paid membership and were confused about what features they received at each membership level.
So they made this chart to clear it up.
And their $1 full membership offer to free-trial members? Yep, thank the survey again.
Their findings showed that while visitors were impressed with all the tools Moz featured, they didn’t know how many would be useful to them. Hence the ridiculous $1 offer to obliterate that objection.
By talking to their customers (and non-customers), Moz was able to test real insights, instead of their own assumptions.
Starting the CRO loop right from the beginning with a good mix of qualitative and quantitative data is how you get to amazing results like Crazy Egg and Moz.
Still An Unbeliever? See How These CRO Experts Use Heatmaps and Surveys for Better Optimization
I asked some of the biggest CRO experts around to give their take on how they use the Crazy Egg + survey combo in their CRO efforts.
Check out what they had to say:
“When you analyze a heatmap, you’ll typically notice that a large portion of your users aren’t taking the specific action you want them to take. By using surveys, you can ask users direct questions that will give you insights on why people aren’t converting through your ideal funnel path.”
“I use surveys to understand intent, and try to understand the barriers for purchase. When heatmaps show heat around images and headings but nothing around content or the main call to action, I have to look back at the answers to uncover questions like, “Do they understand what I’m selling? Are they being distracted by my images? Is it unclear what the next step should be?”
“Asking people what they came to the page to accomplish is my #1 question. Then it’d be a matter of comparing the reasons for people visiting with the areas of the page that are getting clicked. If the heatmap doesn’t line-up with the survey responses, then the page design should change.”
“If the visitors are opting in to answering a survey, you could also ask if they were able to accomplish their goals (e.g., find what they were looking for). Knowing that would give a designer and copywriter even more to go on for improving the page.”
Lance Jones, CMO at Flow
Heat Maps + Segmented Surveys = A Whole Lotta Insights
Whether you’re an ecommerce shop, SaaS startup or digital marketer, you can glean valuable insights from site visitors with intel you simply couldn’t get otherwise.
And with FourEyes surveys, you can supercharge your surveys with advanced data mining at the click of a button, which automatically finds the insights you’d otherwise have to use multiple data crunching tools to get. So instead of looking for CRO gold in reams of spreadsheets or calling your college buddy with the stats degree, all of the juicy correlations are served to you in an instant.
Like the Moz and Crazy Egg case studies show, the best discoveries are waiting with your customers. So add heatmaps to your pages today to see what visitors are doing. Then send out surveys to develop hypotheses as to why they’re doing it.