Mirror, Mirror On the Wall, Which Vanity Metrics Should I Ignore?

by Peter Boyle

Last updated on July 27th, 2017

It’s a big day today.

You log into your analytics and see that after months of hard work you’ve managed to increase your traffic by a whopping 50%!

You pat yourself on the back and head to the pub to celebrate this monumental achievement. But your excitement is short lived. As the subsequent weeks roll on you notice that, despite this incredible leap in traffic, sales have barely moved.

You’ve fallen victim to one of the oldest traps in marketing, the ego boosting vanity metric. 

Sure an increase in traffic is worthy of praise, but when push comes to shove you’re running a business. More eyes on your message doesn’t mean anything if you can’t convert those eyes into paying customers.

You know it’s true and I’m willing to bet that whoever’s calling the shots will be quick to remind you of that fact. Conversion optimization isn’t about increasing traffic, it’s about increasing sales. 

The problem many have is that they associate an increase in any metric as a win. But sometimes an increase in metric X is nothing more than an increase in metric X. It doesn’t help your business and has no wider implications or use beyond bragging rights.

However, there are of course some metrics that you definitely should be tracking. Your job as a smart CRO/marketer is to identify which metrics give data that will help grow your business and which have no broader use or influence.

So let’s take a quick look at vanity metrics and why they should be ignored.

What Exactly is a Vanity Metric?

A vanity metric is exactly what it sounds like. A report or data set that makes you feel good about what you’re doing, but ultimately has no effect on your business or profits. They’re the sort of stats that look great in a press release but don’t help you create a progressive marketing plan.

Time and again I see marketers focusing on increasing various vanity metrics, only to then wonder why they’re not seeing an increase in sales and revenue. Whilst there are plenty of vanity metrics you’ll need to ignore, below I’ve listed three of the biggest offenders:

  1. Traffic levels/Raw page views
    • Traffic on its own counts for very little.
  2. User Numbers
    • Many focus on registered users, which isn’t all that important. Far more important are active users, a segment which will be far smaller than those who are registered.
  3. Social shares/likes/followers
    • Looks great and can bring in traffic. But Facebook likes don’t pay the bills 🙁 .

The problem with the above metrics is that they outline the success of your website, but don’t necessarily represent the health of your business.

Take the below graph as an example.

Vanity Metrics Graph

Image Source

Sure it looks great and, when framed in the just the right way, could give the impression of everything being on the up. The problem that many marketers make is attributing that solitary spike to the action they took directly preceding it. They’ll build their whole future plan around that one action and expect to see repeat gains and explosive growth.

This is the first mistake with relying on your vanity metrics.

Are you 100% certain that you’re responsible for the traffic increase? Looking at the rest of the graph there’s a very good chance that you weren’t. It’s more likely to have been a mention on an authority site, a seasonal change or any number of external influences.

Vanity metrics are anything but stable. There are several external events that can cause huge spikes or steep drop offs with little to no warning. This makes vanity metric data, especially over the short term, unreliable and useless when planning a beneficial campaign.

You might argue that you’ve managed to find killer ideas that have really paid off by looking into your vanity metrics. Hey, I don’t doubt it, but I still argue they shouldn’t be forming the base for your wider campaigns. At best vanity metrics display a correlation between events. Rarely to the help identify the cause.

You might choose to implement a new strategy based on a vanity metric increase and strike it lucky. However, you’re more likely to completely strike out. By relying on vanity metrics you’re treading on thin ice. 

Let’s put it into context with a semi-truth story that also highlights the dangers of relying on vanity metrics over the long term.

Imagine you work for a company who bases its success on user numbers. Thanks to inactive users never deactivating accounts your reports give the appearance of year on year growth. However, you’re a smart cookie and decide to implement a strategy to increase revenue.

Your plan has three major steps.

  1. Cut the dead weight inactive users
  2. Research the active, engaged users
  3. Use that research to roll out a more focused, targeted sales campaign based on your research findings

By better targeting your messaging you should be able to increase customer AOV (average order value), sales and revenue. There’s not a company in existence who wouldn’t want those results.

However, your company has foolishly decided to track a meaningless vanity metric. It’s also the metric which is going to take the largest hit in your strategy.

After implementing stage one the higher-ups panic. For years they’ve seen graphs of user numbers steadily on the climb. Now they’re looking at a graph which reports those very same numbers dropping like a lead balloon.

Straight away everyone is at panic stations. There’s research into what caused the drop and how to get user numbers back up to their former heights. Research which will end with everyone pointing the finger at you.

If however, company X had been tracking revenue from the start, there’d be no panic stations and everyone would be ecstatic at the prospect of your plan.

Vanity metrics don’t just damage your short term research and planning, they skew your opinion of progress. They fill you with false confidence and lead to poor decisions which are ultimately detrimental to your business.

Don’t focus on metrics which have no practical implication. From the start you need to focus on actionable metrics that give useful data and help grow your company.

Identifying Actionable Metrics

As a general rule I like to think of actionable metrics as anything that help you make good decisions. Actionable metrics often identify repeatable processes that are beneficial to your business as a whole.

They answer the important questions such as:

  • How can I increase sales/revenue
  • How can I increase customer lifetime value / AOV
  • What can I do to drive sign ups

This means that you should ignore all of the reports on the default Google Analytics dashboard. Sure the graphs look pretty, and when the blue line goes up you fell all warm and fuzzy inside, but they mean nothing.

GA Default Dash

Yes, That Whole Page is Useless

Instead of relying on the default selection in Google Analytics you need to set up your own goal tracking. The question I’m sure you’re asking is, ‘what metrics should I be tracking then?’.

Well, it all depends on where you are as a business. A startup won’t have the same goals as a multinational. Of course there will be some overlaps but you’ve got to find the unique goals that apply to your individual needs. Below I’ve mapped out common key metrics at different stages of a business.

Startup -> Conversions, Unique Visitor Sources & Social Referrals to Conversions, Acquisition

Small Business -> Conversions, Revenue, Customer Satisfaction, Acquisition, Retention

Enterprise+ -> Conversions, Revenue Per Customer, Retention, Churn Rate

Some of you eagle eyed folk out there have probably noticed the contradictory statement in the above. Yes I have included traffic and social referrals. And no, it’s not a mistake.

Here’s a few tips on how to properly track site behavior and the metrics I’d include in a new campaign.

An Individual Basis

Part of the problem with vanity metrics is that they too abstract. You’ve heard the saying ‘metrics are people’, right? Well, vanity metrics detract from that by giving huge generalizations and sweeping overviews of your data.

Regardless of the metric you’re tracking you need to drill down to an individual or segment specific level. Don’t look at pageviews as a whole. Inspect the difference between new and returning customer’s pageviews. Don’t look at overall customers, inspect customer lifetime. You might see a huge increase in a given month, but if the lifetime is short then you’re going to panic when all of those new customers leave.

Drill down to the specifics and look per segment or individual. It will save you a ton of stress and give you better data to act on.

Revenue

Does this really need an explanation?

Whatever metrics you’re tracking there should always be a heavy focus on revenue. After all, money’s what makes the world go round.

As is always the case revenue stands above every other metric for you to track. You may see a dip in metric X or Y, but if you’re revenue is increasing then all is good. However, if everything else looks hunky-dory but revenue is going down, then you know you’ve got a big problem.

Traffic Source > Conversion (Customer Journey)

As I mentioned above this isn’t a mistake. Tracking your traffic evolves over three distinct phases.

Stage One – Tracking site wide hits which is absolutely useless and gives no good information.

Stage Two – Tracking referral sources. Not bad. Can give you a better insight into your primary acquisition channels and the kind of visitors visiting your site.

Stage Three – Tracking referral sources through to completion. A great way to better understand the behavior and actions of your audience.

By mapping out the whole process from acquisition to final conversion you’ll gather loads of useful data for every stage of your funnel. You’ll identify key traffic referrers which helps understand the kind of audience you’re attracting which in turn should influence how you personalize your copy.

Don’t forget to keep an eye on cross-channel progression!

Don’t forget to keep an eye on cross-channel progression! (Image Source: econsultancy.com)

But more importantly you’ll identify the weakest spots and pages in your funnel. The first step to optimizing your funnel is finding where it leaks and mapping the full customer journey helps plug those holes through both personalization and traditional optimization.

It’s less about understanding the traffic and more about understanding the sources and behaviors of customers. You’re not interested in those who don’t purchase from you, but by understanding the behavior of those who do, you can better optimize your site to attract more of them.

Cohorts

I’ve touched on the importance of segmentation and personalization above, but it bears building on.

Knowing the various segments of your audience and how they react can have a huge impact on your business as a whole. This is where cohort analysis can help. If you’re not familiar with cohort analysis think of it as grouping your customers into segments based on their actions.

Cohort Analysis

Image Source

Cohort analysis helps you identify key lifecycle events. You can understand and identify your repeat purchasers as well as those who registered but didn’t buy or bought once never to return.

You can create a report that shows the behavior of your cohorts over time. If all is steady between different groups there’s nothing significant going on. However, if there’s an unexplained increase/decrease in behavior, it’s time to investigate.

Conversions and A/B Testing

You’re only as successful as your conversion rates.

This is one of the areas where drilling down to specifics really pays dividends. You can’t just say that your site converts 3% of total visitors because it gives no data to improve on that statistic.

Do you amend your landing page? What about your PPC ads, are they the problem? Perhaps it’s your CTAs. Trying to figure out the problem is next to impossible.

You need to set an internal benchmark and test every element and stage to understand what’s working and what’s not. Implement split tests to understand what works and what doesn’t. Separate the tests by funnel stage and you’ll find identifying key problem areas a cinch.

With that being said, never take your eyes off the macro . It’s not unusual for an increase in stage one to two conversions to actually lower overall sales.

Be Smart About Choosing Your Metrics

Don’t be an armchair marketer who thinks that everything going up on Google Analytics’ default dash signifies a great future.

Choose the metrics that are most important to the growth of your business at its current level and drill down to find the key components that make them important. Is it page views that’s important, or rather how new customers interact with your pages?

Drill down, find the key metric and test like hell until it improves.

Vanity metrics set unrealistic benchmarks and will have you chasing your tail. Avoid them at all costs.

7 Comments

DON’T MISS OUT

Get updates on new articles, webinars and other opportunities:

Peter Boyle

Pete Boyle is a conversion focused copywriter and marketing consultant. He helps brands increase their revenue through compelling copy and smart email campaigns. Click here to connect with Pete or download one of his free marketing guides.

7 COMMENTS

Comment Policy

Please join the conversation! We like long and thoughtful communication.
Abrupt comments and gibberish will not be approved. Please, only use your real name, not your business name or keywords. We rarely allow links in your comment.
Finally, please use your favorite personal social media profile for the website field.

SPEAK YOUR MIND

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  1. Mallika Ahuja says:
    August 11, 2016 at 1:53 am

    Vanity Matric is good way for analysis our traffic on site or managing data in right way……….

  2. Anjali says:
    October 6, 2015 at 2:33 am

    @Peter Boyle

    Nice post Peter .
    In A/B Testing, how many conversions do you need per variation for the results to be significant?

    • October 7, 2015 at 6:16 am

      Hi Anjali,

      There’s no one number that signifies significance across the board. It really depends on your site. Most people use tools like Optimizely which helps them understand the significance of their results.

      If you’re not using a tool which automatically calculates the significance of your tests you can use one of the free A/B test calculators online. They’re not as comprehensive as a full service but should get the job done.

      Here’s a few examples;
      http://www.hubspot.com/ab-test-calculator
      https://www.optimizely.com/resources/sample-size-calculator/

      You want to be hitting a minimum of 95% statistical significance, however, don’t rely on that alone.

      Instead of stopping your test as soon as you hit 95% make sure you run it for a longer period of time. I tend to favour month long tests.

      Why?

      Because it gives a better overall data set. You could be seeing an increase in conversions and traffic due to seasonal changes or external influences (such as a reference/link on another authority site). These are short term changes that will likely not translate to an increase over the long term.

      A 25% in conversions over a week long period won’t necessarily equate to a 25% lift in revenue over the year.

      Run your tests for longer periods and always aim for a minimum of 95% significance and you should be fine.

  3. jerome pineau says:
    October 5, 2015 at 3:42 pm

    Thanks Peter – one of the best succinct discussions of doing ROI metrics correctly I have _ever_ read in a long time 🙂
    J

    • October 5, 2015 at 4:50 pm

      You’re welcome Jerome, and thanks for the flattering comment!

  4. Azizul Yusof says:
    October 5, 2015 at 12:59 pm

    Great stuff, as always, Crazy Egg.
    Thank you.

Show Me My Heatmap

Taking @CrazyEgg for a test run @IndiciaNL. First Plus.. Smooth #TagManager implementation!

Rogier | is hier

@Swaaijmans

What makes people leave your website?