Modal Demo

Your AI App’s Homepage Could Be Costing You Millions: In-House Study & Top CRO Experts Weigh In

Your AI App’s Homepage Could Be Costing You Millions: In-House Study & Top CRO Experts Weigh In

Daniel Mowinski Avatar
Daniel Mowinski Avatar

Disclosure: Our content is reader-supported, which means we earn commissions from links on Crazy Egg. Commissions do not affect our editorial evaluations or opinions.

Earlier this year, Crazy Egg put together a focus group of senior buyers. They were shown the homepages of six well-known B2B AI tools (across a mix of industries) and asked what they thought. 

The results?

They were excited about AI features but also felt skeptical, frustrated, and confused. 

It wasn’t uncommon to hear sentiments like “This could help me do amazing stuff” immediately followed by “These claims are totally unbelievable.”

Reason to despair? Not at all.

The opportunity is considerable. If providers can remedy the frustration so many buyers face when interacting with landing pages, they could uncover one of the biggest CRO goldmines anywhere on the big, wild AI-SaaS frontier. 

Our Findings: What Buyers Like and Dislike About AI Tool Landing Pages

Our study sought to uncover how senior buyers—founders, product managers, engineers, and those in similar roles—felt about a cross-sample of six landing pages of popular AI tools. 

It set out to answer three questions:

  • What is motivating decision-makers to consider buying AI tools?
  • What do decision-makers like about the homepages of AI tools?
  • What do decision-makers dislike about the homepages of AI tools?

Because the research team was particularly interested in uncovering why buyers felt the way they did, both qualitative and quantitative data were gathered. 

Methodology

  • The team conducted 33 usability sessions.
  • The homepages of the following companies were shown to users: Lovable, V0.dev Unusual.ai Jasper, BrazeAI, Copy.ai. 
  • All remote unmoderated sessions were conducted via UserTesting. 
  • The participants held one of the following roles: founder, entrepreneur, product manager, engineer, developer, or marketer. 
  • Participants were asked the same questions about the homepages, covering their interest in features, the clarity of copy, and the likelihood of purchase.

Why Are Decision-Makers Considering AI Tools?

Participants were first asked about their top motivations for researching AI tools. Cutting development and coding costs (56%) was the main reason, followed by help with composing marketing communications, such as SEO copy and newsletters (36%). 

Personalization scored modestly (33%), which fits with third-party research that it is a promising but somewhat overhyped use case. Surprisingly, only 25% of users were actively researching tools to save time.

What Do Decision-Makers Like About the Homepages of Popular AI Tools?

Next, buyers were asked what they thought about six homepages of AI tools across three areas: time-saving potential, features, and overall success at generating interest. 

Nearly all said they recognized that the apps could help them save time (97%). Most homepages described at least one helpful feature (70%), and just over half of the homepages succeeded in evoking curiosity about general AI functionality (58%).

What Do Decision-Makers Dislike About the Homepages of Popular AI Tools?

It was when buyers were asked what they thought about the clarity and believability of the homepages of popular AI tools that the conflicting nature of their responses became apparent. 

97% of respondents felt the homepages lacked a clear explanation of how the product worked. Nearly three quarters (67%) were frustrated by the absence of in-product examples (like dashboard screenshots) and real-world use cases. And more than half (57%) said they didn’t trust the claims.

Why Are AI Tool Homepages Evoking Such Mixed Reactions?

Buyers are actively looking for new AI tools; they’re intrigued, and they see the potential. Yet they’re also confused, frustrated, and untrusting. 

Only 45% of the buyers Crazy Egg spoke to said they would definitely try the AI tool they were shown from the homepage. At the same time, 97% said they saw potential benefits. 

We need to ascertain why buyers feel this way. What specifically is turning them off so much? 

And that’s precisely where the qualitative answers come in. 

Here are the main objections drawn from the transcripts of the focus group participants, with exact quotes.  

Lack of specificity and transparency regarding features

“I see a lot of stuff here where you’re trying to tell me about the cool aspects of it, but I want to see under the hood. When I sign up for $500 a month, how do I actually use it?” Computer application specialist

“I don’t know how it integrates, for example, with a shopping module. How can I host this? How can I just get notified? How are all of these APIs going to work?” Developer & entrepreneur

Lack of differentiation and clear positioning 

“Everything now is AI-powered. And for some reason, that is supposed to be a big deal. There’s a lot of competition.” Creative director

“What makes it special? Why should I give you my money?” Project coordinator

Overuse of sales jargon 

“What really frustrated me is that I was trying to figure out some acronyms. It completely ticks me off because I didn’t get it immediately.” Founder & entrepreneur

“There was a lot of promotional content, but there wasn’t a lot of highlighting of use cases.” Marketer

“The header ‘What can I help you ship?’ is confusing for people that may not be familiar with that terminology.” Developer

AI aversion and mistrust

“I am not a huge fan of AI. I feel like it just kind of takes away a lot from human creativity”  Founder & entrepreneur

“I wish I could try a free sample just to see how it turns out, then I would probably have a better chance of signing up.” Marketer & product manager

Concerns about the quality of AI outputs

“I’m a little skeptical. It sounds impossible to build a website or an app just from writing in a prompt. What would the quality be at the end of the day?” Marketer

“The big caveat is obviously the quality of AI—if it can actually produce a decent quality of work.” Marketer

“I want to know why this is better than ChatGPT.” Founder

Overwhelm at the number (and apparent similarity) of AI tools

“We’re constantly bombarded with AI tools and new technologies, and we have to try all of them. And we have to test them extensively before we can implement any solution.” Director of IT company

“It doesn’t clearly explain that it is an AI tool and what it can do for me.” Entrepreneur

Mistrust of testimonials and client results

“The testimonials sound like fake quotes.” Marketer

“Some of these numbers, like a 3x increase? How do you measure that to know that it was an increase?” Head of marketing

“I wanted to see some realistic true stories.” Content creator & founder

How to Seize the Opportunity of Bad Homepages: An Expert View

Software companies—whether they’re AI-native or “traditional” providers integrating AI features—could be losing millions because of their homepages.

It’s well evidenced that homepages are critical for selling software. Both indirectly—as an influential component of complex buyer journeys—and in the immediate term, for ready-to-buy visitors. What’s more, 76% of SaaS tools now offer some form of AI functionality, according to research by Panintelligence.

Consider that mid-market SaaS companies ($10M–$100M) have, on average, 5000–15000 monthly visitors and that 20% of visitors go to the homepage. The average lifetime value of a SaaS lead is between $3000 and $15000

What would a 0.5% increase over the course of a year mean? An additional $180K to $2.7M of value. For a 0.5% boost. 

Most AI-SaaS companies—especially bootstrapped startups—are sitting on a colossal opportunity. 

To help me understand the nuance of the AI landing page problem at a deeper level and put together a solution, I spoke to seven leading CRO and copywriting experts. 

Our expert contributors
Talia Wolf: Founder and CEO of GetUplift and author of Emotional Targeting. 
Lars Lofgren: Growth advisor to brands like Automattic, NP Digital, and I Will Teach You How to Be Rich.  
David L. Deutsch: Acclaimed copywriting strategist who began his career at Ogilvy & Mather on Madison Avenue and has generated over a billion dollars in sales for his clients.  
Anthony Pierri: Co-founder of Fletch and all-round landing page genius.  
Amy Posner: Copywriting coach and business growth mentor who runs The Breakthrough Beat newsletter. 
Bryan Collins: AI educator and founder of the AI Flash Report.
Brian Massey: Conversion expert and managing partner at Conversion Sciences and author of Your Customer Creation Equation.  

When I began connecting with these experts, I partly expected a reiteration of good conversion and copywriting principles. And they did provide these in abundance. But that wasn’t all I got. 

What surprised me was the degree of creative adaptation needed for AI tools. It requires much more than a copy-and-paste approach. 

1. Honor the AI Hype Cycle

AI is not hype. But it is overhyped.

Lars Lofgren calls it “one of the biggest hype cycles in corporate history.” And this simple truth goes a long way in explaining why companies are getting it wrong with their homepages, eliciting the mixed responses uncovered in the research. 

“It’s created a situation in which leaders—and CEOs are terrible for this—are rushing to meet perceived expectations. They’re driven by urgency and FOMO and not taking the time to figure out what customers actually want.”

The hype cycle isn’t just speculation either. There’s solid data behind it. According to Gartner, we are now entering the Trough of Disillusionment after a period of inflated expectations. 

Lars is quick to point out, however, that this doesn’t mean you should stop talking about AI. To the contrary, most companies should be putting it front and center. Just not in the way you might think. 

“If you’re not shouting about AI, you’re irrelevant,” he says. “We had a situation when I ran marketing for KISSmetrics, an analytics startup. It was all ‘real-time data, real-time data’ from customers. Everyone wanted it. We looked into it, did deep user research, and realized it was all hype. People said they wanted real-time data but never actually used it.

“So we cancelled the feature rollouts, and we didn’t run with them on the basis that real-time analytics was hype and users didn’t really need it. We were right. But it harmed the company in a big way as we kept losing customers to our main competitor, Mixpanel. Sometimes you just need to give people what they’re asking for even if the impact is minimal.”

The enemy here is a mismatch between corporate and consumer expectations, and this matters for landing design for two reasons: 

  • It points towards the need to cater to expectations, not just internal assumptions of value. You should be led by what your customers are asking for, even if that’s not where you believe the real value is. For AI-native software, it’s a case of prioritizing the most in-demand features. For non-native companies, it’s about which AI augmentations you give attention to. 
  • It highlights the necessity of ongoing customer research. The AI features you need to prioritize can change quickly. Hype cycles and shifting customer expectations go hand in hand. Research needs to be ongoing so that, as disillusionment sets in and preferences change, you can adapt quickly. 

The buyer committee is looking for a certain set of features, so it’s important you account for these. And the fact your product isn’t AI native doesn’t mean you’re immune. But don’t assume that the most valuable features are necessarily the ones you need to sell. 

2. Base Landing Pages on Good Qualitative Data

So far, so good. You’ve accepted the need to prioritize the features that buyers actually want. 

But how do you go about uncovering these buyer priorities?

For Lars, the answer is simple: “All the good old-school stuff. Customer surveys—keep them open-ended—feedback forms, one-on-ones, sales transcripts. Spend two or three months gathering this stuff and you’ll have more than enough to go off.”

Coincidentally, one thing I promise you won’t find during this process is your target customers looking to “elevate,” “amplify,” or “unlock” anything.

However, there is a caveat.  

In his book Your Customer Creation Equation, Brian Massey explains, “Even if you have done extensive surveying, focus groups, and interviews, you most likely don’t have an accurate picture of your web visitors…people don’t really know why they do the things they do. In fact, they will often do the opposite of what they claim. So, if you can’t use your research, what should you do? Treat the research as hypotheses, not answers.”

Your research forms the basis for later testing. It gives you an initial idea of which AI features and functionality your target market is genuinely interested in. It is a necessary first step, but it only serves as a hypothesis, especially in a context of AI tools where so little is known. 

A four-pronged approach to market research covers all these bases: 

StageGoal Technique
Analysis of existing customer baseTo identify emerging needs and AI preferences of your existing ICPsVisitor surveys
Email surveys 
Analysis of prospective marketsTo test product-market fit with new AI-buyer segments Surveys via third-party servicesPaid traffic
Deepening understandingTo understand the needs of best-fit customers at a deeper levelOne-on-one interviews
Competitor analysisFinding out about AI features gaining traction that you’ve missedSocial listening

For non-native AI companies, it’s also important to avoid immediately jumping on the AI feature bandwagon. Early-stage surveys provide important insights, but insight into the real needs of best-fit customers, and their associated AI preferences, will likely only come from one-on-one interviews. 

3. Pick Your Positioning Lane, Clarify It, and Stick to It

Frustration at poorly explained features was the primary issue raised by our group of buyers. However, overall lack of clarity around what tools did—especially in relation to other products—was a close second. 

Poor positioning is one of the problems Anthony Pierri, partner at Fletch and one of the world’s most respected homepage experts, sees most often. “‘People read our website but don’t understand what we do’ is by far the most common complaint we hear from venture-funded B2B software companies.”

To clarify where you stand in the market, nothing works quite as well as a succinct positioning statement. This provides a targeting template not only for your homepage copy (above the fold, I’m looking at you), but for all your landing and feature pages.

It also helps you avoid generic pseudo-marketing speak like in the example from Braze below. What is that ™ symbol trademarking? I’ve no idea.

For Anthony, positioning is the key to communicating value: “Whatever the specific cause of confusion, the common thread is weak positioning. Getting positioning right makes it so your top-fit customers will immediately understand your value—with no confusion.”

He recommends a positioning statement informed by two core ideas:

  • Positioning anchors: Reference points you use to make sure customers know exactly what the product is and who it’s for. 
  • Unique value: The specific problem related to one of those anchors and how you solve it in a differentiated way.

Primary positioning anchors use familiar concepts to help an audience “mentally classify” what a product does. This can be a product category, use case, or competitive alternative. A secondary anchor then adds descriptive context. It provides nuance. This might be a department or a desired outcome, for instance. 

In the example below, Jit (one of Fletch’s clients) clearly states its product category—security AI agents—with a definite outcome—”tasks done, not just flagged.”

Your unique value, the second part of your positioning statement, describes your customer’s problem and your differentiated way of solving it. It is a problem tied to an anchor—”CRMs (product category) are too unwieldy (problem)” or “Using AI to speed up customer service (use case) is complex and costly (problem)” for example.  

You then build on this unique value by describing the differentiated way you solve this problem, usually by citing a core product feature or USP. For an AI product, this will correspond to the research you have done into which features users are most interested in and excited by. 

Remember, this positioning statement isn’t necessarily your copy. It’s a tool for guiding and clarifying when you come to create your landing pages, and it will help you avoid the ever-present trap of ambiguity. It’s an indispensable reference point. 

4. Identify Emotional Problems 

Let’s return for a moment to a telling statement from one of the buyers that participated in our survey: 

“Everything now is AI-powered. And for some reason, that is supposed to be a big deal. There’s a lot of competition.” 

For Talia Wolf, widely recognized as one of the world’s top CRO experts, the risk of “leaning heavily into generic themes” is particularly great when it comes to AI products. The basket of benefits—automation, scalability, cost savings—are largely the same between products. 

For Talia, buyers are fundamentally emotional, and talking in emotional terms provides a sure route out of the generic. “Whether you’re selling toilet paper, AI tools, cyber security, or cooling systems, at the end of the day anyone coming to your landing page is there to solve a particular problem they have,” she told me. “Your goal is to make sure people can immediately see the pain you’re going to lessen or eliminate for them.”

Teamwork, a client that Talia has worked with through her company GetUplift, excels at emotional targeting, as in the example below, which focuses on the “headaches” and “pain” of clunky project management software.

Three questions (drawn from Talia’s nine-step Emotional Targeting™ Audit) make up a perfect antidote to the hyper-logical techno-speak that dominates the landing pages of so many complex AI tools. 

  1. Can my prospects see their pains reflected in every step of the journey?
  2. Can my customers immediately (and clearly) see what’s in it for them on this page?
  3. Am I using stories that resonate?

These questions should form part of every landing page creation iteration process. They act as a mini checklist to ensure that all necessary emotional components—pains, promises of resolution, connection, gratification, and social proof—are all present. 

5. Pay Careful Attention to the Language Your Prospects Are Using to Talk About AI (Even If It’s Wrong)

Jargon—our buyers hated it.

The mainstream AI lexicon is woefully inaccurate. Certain groups will happily use the term “AI” to refer to everything from a conversation interface to a virtual BDR. For others, the differences between a decoder-only LLM and a decision tree are as obvious as the sun shining in summer. 

In most cases, you will be talking to one of two categories: 

  • Non-tech B2B professionals: These people don’t have any technical training and rely on a “folk” AI vocabulary. They may, however, use terminology specific to their field (doctors, scientists, academics, etc.).
  • Tech buyers: Segment-specific tech buyers (engineers, coders, data scientists) with formal training in AI tend to prefer to use a precise AI vocabulary. . 

It is important that you pay attention to the AI “dictionary” of your target audience in the research phase of landing page design, with interview scripts a particularly valuable source of information. You then need to mirror this across your whole homepage

Keep in mind, however, that technical language is not jargon. You can speak in plain English while being technical (as in the example below). 

For copywriter David L. Deutsch, talking about AI in sales-speak is one of the most widespread and insidious of copywriting mistakes: 

“Write to these people not as someone trying to sell them something and not using a bunch of marketing pablum they’ve heard in the past for software and AI products, but simply. Take your understanding of the prospect and customer at a deep, non-superficial level, and write as if speaking across the bar to a good and valued friend.”

6. Sell With Truth (Your Future Customers Aren’t Idiots)

I want you to try an experiment. 

Open up a doc and write down everything your AI tool doesn’t do

You now have a list of all the features, and associated claims, that you shouldn’t go anywhere near. You might even draw attention to these limits in your end-of-page FAQs. 

Annoyance at unbelievable claims surfaced again and again during our focus group sessions and was the third most cited issue with homepages. Consider the example below—the only end-to-end sales solution, really?

Fortunately, there’s a simple way of remedying this problem: selling with truth. 

For copywriting coach Amy Posner, this is the most powerful way of alleviating doubt in a smart buyer. And make no mistake about it, your buyers are smart. 

“There’s always been advice about writing to the ‘lowest common denominator’ or to a ‘low level of education.’ I don’t buy it. Write to your buyer like they’re smart, and they know how to make a decision.” 

Straightforward, specific messaging alongside claims that can be backed up can work incredibly well, as in the example below (notice the little “i” next to the “#1 AI Video Platform” claim).

For Amy, absolute transparency is the best way of putting skeptical prospects at ease:

“Don’t just show them the wonders of what it does. Tell them what it doesn’t do, while telling them how they’ll benefit from what it does offer. Be honest about what people are afraid of; be transparent. Telling the truth builds trust. Hype doesn’t, and AI is a product that’s had a lot of hype!

“We’re far enough along in the AI journey now that most have seen, experienced, or read about the good, the bad, and the ugly. Transparency will be welcomed. It’s a minefield out there, and buyers are smart enough to discern what’s real. So, sell them with the truth.”

7. Set Up Ongoing Testing Infrastructure

While writing this article, I sat down with David L. Deutsch, a true doyen of the copywriting world. We discussed the unique issues of writing persuasively about AI and the problems highlighted by the research from Crazy Egg.

Something that stood out to me over the course of our conversation was how the copywriting practices used by the SaaS companies I’ve worked for over the years depart from the rigor of traditional direct response marketing.

When working at Ogilvy and Mather on Madison Avenue, he described how copywriters would constantly iterate against a top-performing control, with ongoing variation testing often yielding unexpected results for sales letters and ads. 

For David, testing in the era of AI—when we’re still learning what does and doesn’t work—is more vital than ever. Assuming visitor numbers are large enough to provide statistical significance, he argued that testing should be built around five key principles. 

  1. Testing should be ongoing: Successful optimization is a process of generating lots of small wins over a long period. 
  2. Focus on key decision enablers, not minutiae: Your subheader halfway down the page likely isn’t moving the needle. Focus on CTAs, hero-section value props, and your overall emotional angle. 
  3. Understand that conversions don’t necessarily equal sales: What David calls “front-end lift” means very little if it doesn’t lead to paying customers, or if it counteracts other profit boosters (like upsells). Always look at conversion testing through the lens of overall ROI. 
  4. Go beyond A/B testing: The true value of testing lies in multivariate testing, not simple A/B tests. 
  5. Pre-tests show you what works: If you’re sending paid traffic to your landing pages for the purposes of testing, quick-fire Facebook and LinkedIn ads can give you an initial idea of what resonates.
  6. Embrace the AI CRO Tech Stack 

Testing practices from the world of traditional direct response copywriting can take your conversions into the upper atmosphere. Combining them with an AI-augmented stack will lift them into the stratosphere. 

According to Brian Massey of Conversion Sciences, a comprehensive stack can help you run tests—and implement results—faster and with much greater confidence. All of which are vital in an era when buyer preferences, driven by a rollercoaster AI hype cycle, are changing faster than you can blink. 

He recommends a five-part AI-augmented CRO stack that covers the following areas: 

  • Automated page evaluation: An AI examines a page and suggests optimizations. These tools automate the work of a junior conversion optimizer. Examples include Fibr.ai and CROBenchmark.
  • AI-driven testing: Using AI to reduce the traffic required to run A/B tests or multivariate tests. Evolv.ai falls into this category.
  • Behavior-based personalization: An AI monitors the behaviors of visitors and puts them into a category for personalization. AB Tasty is one of the leaders in this category. 
  • AI-generated content: An AI generates alternative variations of content and tracks results. Most AI-augmented AB testing tools will provide this.
  • AI-generated data analysis: An AI evaluates the analytics of a website—including heatmaps—to provide ideas for testing. SiteWiz.ai offers this functionality. 

If you’re new to AI-assisted CRO testing, you might feel overwhelmed at the prospect of building a tech stack that accounts for all the areas described above. However, it’s important to keep in mind that these tools aren’t designed exclusively for conversion professionals. The best way to learn how to use them is through simple trial and error with your own experiments. 

Oh, and Lock Your CEO In the Boiler Room Until After Publication

I’m not generally in favor of imprisoning CEOs. When it comes to landing pages, however, I make an exception. One experience kept cropping up as I spoke to people for this article: tech CEOs have a tendency to ruin good copy. 

CEOs, if you’re reading this, the best thing you can do is say your piece, give the process to the right people, and get out of the way. 

You might think “Up-levelled AI-enablement for your SQL pipeline KPIs on autopilot” sounds good. 

It doesn’t. Let the professionals do their job. 

Pulling It Together: Honesty, Emotion, and Technical Transparency

Theory is good. 

But what happens when we weave all of the advice above into a working template? 

Well, that’s exactly what I’m going to do.

I want to give you a practical tool. One that you can take out into the world to begin an AI homepage revolut—ahem, iterative optimization. 

The template below is a starting point. It uses an imaginary AI cold outreach (AI BDR) platform as an example. Expect it to change as you run your own tests and iterations. 

Here’s an explanation of why each feature is included and how it fits into the broader strategic picture. 

Homepage SectionExplanation
The crystal-clear hero (above the fold)The aim here is to clearly state where you sit in the category and how you’re differentiated in language your audience understands. Use your positioning statement to inform your headline and any additional explanatory text. 
The emotional problemYou should describe the main emotional problem that your research has shown. 
The technically informed solutionThis section provides a solution to the emotional problem described above with a technically informed solution. 
Five to six top features (based on research)Feature hierarchies should be based on customer research. For non-AI native tools, they should be woven into existing features according to preference. 
Cross-section of testimonials with dataWith testimonials, focus on one tangible result (if available) for each review, provide a cross-section of customers, and include pictures. 
NecessitiesThis is where you quickly cover the features that are logistically non-negotiable for customers. It’s a box-ticking exercise. 
FAQsDrop-down FAQs should be based on the questions identified in customer research. 

A Homepage Could Be Losing Millions, Really? 

Bad B2B homepages have been a problem since the dawn of the internet. 

Ambiguous messaging, outlandish claims, hyper-rational messaging—you would be forgiven for thinking it’s all old news. 

What’s unique about AI tools is their ability to accentuate these problems. 

All of the hype, new jargon, CEO overinvolvement, unclear differences between emerging product categories, and general lack of customer research have come together to create an industry-wide problem. It’s a near-perfect “just-add-water for a crappy homepage” mix.

However, look beyond the problem, and it’s impossible to ignore the opportunity. 

There’s no better time to do something sensational with your homepages. All while your competitors continue to misunderstand, frustrate, and confuse. 

Remember, even a 0.5% uplift in homepage conversions can mean millions in extra revenue. 


Scroll to Top