3 Critical Sources Of Qualitative Feedback You’ve Been Ignoring

by Sid Bharath

Last updated on January 9th, 2018

Conversion rate optimization isn’t just about continuously running A/B tests. There’s a science to it. You collect data, make hypotheses, and then test them. A/B testing is just one part, but it all begins with the data you collect.

The more data you collect, the more informed your hypotheses will be. In fact, trying to act on limited data might lead to misleading results. You need both quantitative and qualitative data to come up with a meaningful hypothesis.

For example, let’s say Google Analytics shows you that lots of people are dropping off from your pricing page. You have very little data to go on, and you could test everything from copy to button color without fixing the true problem.

With a tool like Crazy Egg, you get more data about what parts of the page they are looking at, where they are clicking, and how far they scroll. Now you can come up with better tests, but you still don’t have the full picture.

The final piece of the puzzle is actually asking customers why they drop off. Is there something that turned them away, or were their questions not answered? You can call them up for direct interviews, or use other methods like online surveys.

However, direct calls may not always be feasible or convenient. And surveys may give you limited or biased answers. Fortunately there’s already a lot of customer feedback available to you. You just don’t know it yet!

In this post, we’ll look at 3 sources of qualitative feedback you’ve been ignoring, and how you can make use of them.

1. Social Networks and Forums

Social media is not just a marketing channel. It’s a great place to find feedback from customers and potential customers. These days, people expect customer service even on Facebook and Twitter, and they’re voicing their complaints there. You just need to listen!

A couple of months ago I saw a tweet from someone about there being two pop-ups at the same time on my site. Now, obviously I didn’t intend to do that on purpose but I may not have found out about it unless I was watching Twitter. If I had missed it, who knows how many people that would have been turned off by my site. No analytics software would have given me that answer.

People are talking about your site, your products, and your industry everywhere. Even if it’s not directly about you, it might still be pertinent. After all, your job is to solve your customers’ problems, even if those problems aren’t specifically with your site.

Keep tabs on social networks, online forums, communities, chat groups, and anywhere else people might be talking about their complaints or challenges. When you hear feedback about you or your industry, listen to it and take action.

To make the process easier, use some of these tools:

  • Google Alerts – With Google Alerts you can track any keyword and you’ll get an email whenever that keyword appears online. Use it to find brand mentions on other sites or blogs.
  • Social Mention – For mentions on social media, the aptly named Social Mention gives you real-time alerts. They track user-generated content across hundreds of social media and bookmarking sites, so you’ll know when someone talks about you on Twitter, Digg, YouTube and other sources.
  • Notify – This is a pretty nifty tool if you use Slack. It tracks mentions on Hacker News, Medium, Reddit, and many other forums, and sends them straight to a Slack channel. I personally use this for alerts about brand mentions and competitors.

Pro Tip

The Internet is such a vast place that it’s easy to miss things, or to get overwhelmed. As your company grows, monitoring public feedback becomes a team effort. Use the tools I mentioned to collect everything in one place, and then assign people to organize and make sense of the data to extract actionable insights.

Dairy Queen has an entire department dedicated to listening for feedback online and offline. All the feedback goes into one big pot and someone reads every single comment. Sometimes it might be a one-off complaint, but other times there’s a pattern in there that points to a larger problem.

2. Reviews and Review Aggregators

For ecommerce stores, a very obvious source of qualitative feedback is product reviews. However, it’s largely untapped because most businesses only look at their own reviews. They tend to ignore reviews on competitors’ sites, and marketplaces like Amazon and Ebay.

Amazon’s product listings usually have a customer Q&A section, followed by reviews from real customers. Looking at the comments, even for products from competitors, can give you a good idea of what questions consumers have when they shop for those products.

Here’s an example of a shoe on Amazon. There’s a question about shoe polish, and then in the comments you can see some complaints about the fit. Even if you don’t sell this particular shoe, you can add this information to the product pages on your store and pre-empt your shoppers’ concerns and objections.

amazon qualitative feedback

amazon qualitative feedback

For software businesses, there are review aggregators like GetApp and Merchant Maverick, not to mention numerous bloggers and affiliate marketers who write reviews about the products they use.

A couple of days ago I was reading a review of Long Tail Pro on Matthew Woodward’s blog. He was reviewing the Platinum version of the latest release of the product, but the website only had pricing for the Pro version. I held off on my purchase but, when I looked through the comments on his review, I found that the Platinum version comes as an up-sell only after you buy the Pro version.

Now, if someone from Long Tail Pro is watching, they might want to take note of this. Clearly there is some confusion about the two versions and it’s not obvious on the Long Tail Pro site. There were quite a few comments about it, but there are probably many more who visited the site, couldn’t figure it out, and then decided to look at something else.

Furthermore, the review also sparked off discussions about other similar products and what features customers care about the most. Again, this is extremely valuable because businesses in the industry can then tweak their website and highlight those features.

3. Customer Support

This is the most obvious source of feedback, but it’s often overlooked, especially if the customer service teams are not well integrated with the rest of the business. All those customer support emails and calls contain heaps of valuable feedback that you aren’t taking action on. Think about this the next time you decide to outsource your support.

It’s not enough to just answer customer questions because if more than one person has a problem with something, there’s a deeper problem at hand. Like with the social network feedback, analyzing your customer support tickets will help you identify problems on your site that your analytics can’t.

daFlores is an online retailer of flowers. To increase conversion rates on their site, they hired Conversion Rate Experts. As part of the initial research, they asked the customer support team to create lists of the most common inquiries.

What they found was that many customers were worried that their flowers would not be delivered on time. These conversations happened on their support channels, and no one had given it a second thought until now. While it doesn’t seem like a big deal to assure customers that their flowers will arrive on time, the problem is that for every one who calls in, there are 10 others who don’t and leave the site instead.

To solve this, all they had to do was make it clear on the homepage that daFlores had a same-day delivery program. They added a banner with a countdown timer to tell visitors to order quickly in order to be eligible. This assured customers that their flowers would reach on time, and added urgency to motivate them to buy. The result was a 27% increase in conversion rates.

daflores qualitative feedback

Pro Tip

Make sure that every customer support email or call is being tracked and stored in a central place. Use a helpdesk tool like HelpScout or Desk to manage this. Then, periodically go through them to see if there are any overarching trends like the daFlores example.

Start Combing

I won’t lie to you, there’s a lot of data out there and it will take you time to comb through it all and extract feedback from it. But it will be worth it because you’ll find things you never thought about before.

In fact, it will save you all the time you’re going to waste running meaningless tests. So go have a look at these sources and let us know what you find!



Get updates on new articles, webinars and other opportunities:

Sid Bharath

Sid is an entrepreneur, growth hacker and writer. To find out more about him or get in touch, check out his personal site.


Comment Policy

Please join the conversation! We like long and thoughtful communication.
Abrupt comments and gibberish will not be approved. Please, only use your real name, not your business name or keywords. We rarely allow links in your comment.
Finally, please use your favorite personal social media profile for the website field.


Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  1. Amad Ebrahimi says:
    September 13, 2015 at 1:07 am

    Great article Sid and thank you for the Merchant Maverick shout-out.

    You couldn’t be more correct about qualitative data. This is an area that I’ve been trying to pay more attention to. Your third point (Customer Support) is actually what I’m focusing on right now, so this is definitely a timely article.

    Usability testing is another great source of qualitative data.

    Take care,

    • Brian says:
      February 6, 2017 at 4:18 pm

      Hey Sid,

      Very insightful post, it as made me pay a closer attention to our feedback from around the web.

      Amad as a point “Usability testing is another great source of qualitative data.”

Show Me My Heatmap

I’m just @crazyegg’ing everything these days.

Kyle Mitchell


What makes people leave your website?