My first real introduction to Larry Kim was when I helped coordinate the webinar: The 10 Weirdest A/B Tests Guaranteed to Double Your Business Growth.
Larry came up with that genius headline and naturally we had a full house that day. I’ve been hooked on Larry ever since.
We wanted to catch up with Mr. Kim and ask him a few questions around conversion rate optimization, testing, and digital marketing.
Here’s what he had to say..
1. With A/B testing, most variations of the control underperform and fail. Correct? And if so, why is that?
Yes. This is true and becomes increasingly the case the more you do A/B testing.
Here’s why: Conversion rates aren’t random. There is a theoretical limit to how high say, a whitepaper download form can convert at. It’s not like you can keep doubling the conversion rate forever.
This is essentially the law of diminishing returns. Every time you discover a new “winner,” the harder and harder it becomes to find the next winner and the extent to which the new contender beats the previous champ becomes smaller and smaller.
You can read about this and other CRO Truth Bombs here.
2. In the modern web, a smooth experience is becoming very important. A/B tested websites often feel inconsistent and sometimes ruin the overall “experience.” Is this a legitimate concern? If so, what are some ways to avoid this?
Some advanced users might be annoyed at seeing different homepages or unusual URL parameters based on different browsing sessions. It bugs me a bit but probably just because I’m an internet marketer and so, of course, I notice that. My wife is a doctor and probably wouldn’t even notice.
3. Can you share one of your biggest marketing fails? What went wrong? In hindsight, what would you do differently?
There are too many to list. But off the top of my head, the two that come to mind are:
- I was about three years late to using organic social media (e.g.: Twitter, Facebook, LinkedIn)
- I have yet to do anything interesting related to video, YouTube or Instagram
4. Can you share one of your greatest marketing wins? What happened? In hindsight, what would you do differently?
— Larry Kim (@larrykim) December 20, 2016
In hindsight, I would have made it three years earlier. That’s the problem with great ideas – they’re usually obvious in hindsight.
5. Have you ever been involved with an A/B test that sought to optimize for something other than business metrics? If not, can you imagine a legitimate reason to do so?
Say you really wanted to expand your market to a new audience – you could optimize for new visitors (as opposed to repeat visitors) – even though the conversion metrics for “new” people is usually much worse than people who are familiar with your brand.
I also often advise people to optimize for user engagement metrics like Click Through Rates, even though they are technically just one step removed from a business metric like Cost Per Conversion.
6. If I had an online business, and I gave you $1 million to spend on marketing, what would you spend it on? Without knowing what type of online business it is?
Probably Facebook Ads. And you wouldn’t even need 1 million.
7. Is there a new, emerging marketing tactic you’re really excited about?
These are not necessarily new, but are exciting and have new features being added every week:
- RLSA & Audience Targeting for Search Ads
- Messaging / Chatbots
- Twitter Ads & Facebook Ads
- Account Based Marketing
- Blogging on Medium
- Content Promotion Networks like Taboola, etc.
8. Is A/B testing here to stay? Do you envision it being replaced by something else?
A/B testing is likely to be pushed down a level in the marketing stack. If you’re just testing small changes to flows or presentation, an AI-enabled system should be able to do this monkey work, automatically.
As an example, in Facebook Ads, you can’t even A/B test your ad creatives. The platform auditions them for you and automatically picks a winner. In Google AdWords, the platform can even re-write your ads for you if it thinks your ad copy is poor. The same will happen to landing pages (which are essentially just bigger ads)
That is not to say that other important marketing disciplines like User Experience, Design (etc.) are any less relevant today than before. But the mechanical grunt work behind A/B testing is likely to become increasingly automated.
9. If you could describe modern digital marketing in 3 words, what would they be?
Build Your Brand.
10. Out of all the songs that have ever been written, if you could’ve authored one of them, which one would it be?
Forever Young by Bob Dylan
11. The internet (and internet marketing) is constantly changing. What do you do to stay current?
I do a lot of marketing. Most marketers, over time, get promoted to managerial roles that are less hands on. I still spend a good part of my day doing internet marketing activities like blogging, SEO, Adwords, Facebook Ads, content marketing, etc.
12. After years of testing and experimentation, are you starting to notice repeatable trends in human behavior? Predictable behavior? Or, are you always surprised?
Earlier on in my career, I believed the CRO Kool-Aid – that small A/B tests, when added up, could make a big difference. Naturally, I would want to attribute changes in results to my hard work in conducting these experiments. But now that I’ve been doing this for over a decade I realize that I may have been misattributing cause and effect. The biggest CRO hack by far is brand awareness. That people, consciously or unconsciously, dramatically favor clicking on and buying from the brands they have heard of and love.