The A/B and Cs of Email Campaign Testing

by Kathryn Aragon

Last updated on August 7th, 2017

The foundation of conversion rate optimization is testing…

Testing your offers.

Testing your messaging.

Testing your web pages and other touch points.

Today, we look at how you can test your email campaigns, with an infographic, courtesy of EmailMonks, that walks you through the process.

But first, secure your seat in our webinar today, featuring Conversion Sciences’ Brian Massey:

Join us for a Crazy Egg Hard-Boiled Conversion Webinar

brian massey
Featuring Brian Massey, Conversion Sciences
May 28, 2pm US Central Time
Sign up here (it’s free!).

How to Reverse-Engineer a High-Conversion Landing Page

Now, let’s look at how pro email marketers test their email campaigns.

Email Campaign Testing: Choosing Between the A/B and Cs of Email Marketing Elements

04 - email monks - ab testing emails

Let’s Review

A/B testing, also known as split testing, compares two versions of a single variable to find the version that yields the most opens or clicks.

The same rules apply, whether you’re testing a landing page or your email clickthrough. Read here for 6 tips for more successful A/B testing.

For example, one email has two versions, each with a different subject line. Or one email has two versions, each with a different CTA. Half the list is sent to one version, while the other half of the list is sent to the other version.

This tells you which version is more successful at getting your desired goal, whether that’s a higher open rate or a higher clickthrough rate.

7 tips for getting better results

1. Keep it simple!

Focus on one variable at a time. Look for the elements that could yield your biggest wins, and test them first: subject line and call to action.

2. Keep your test campaigns consistent.

Always compare apples to apples. If you’re testing headlines, come up with a hypothesis for types of headlines your subscribers will click on. Then create unique headlines for testing your hypothesis.

3. Determine your end goals before you start testing.

Know what your goal is: higher open rate, for instance. Make sure your test will help you achieve your end goal.

4. Check your pre-testing elements.

There’s no need to reinvent the wheel. Review old test results before setting up your current test. Knowing what worked and what didn’t in previous tests can help you get better results in this test.

5. Frame the test.

One element. One test. Don’t try to rush results.

6. Split the list.

Your list needs to be large enough to run an A/B split test. If it’s very large, you may run your test on only a small test segment. If the test gets good results, you may send that email to your entire list.

7. Change the elements.

If a test yields poor results, try a new version or test a different element. Each test you run should give you more information for the next test.

The idea, friends, is to be always testing.

9 elements to test in email marketing

  • Call to action
  • Personalization
  • Subject line
  • From line
  • Design
  • Time and day of send
  • Type of offers
  • Email copy
  • Rich v. plain text

Advanced email testing tips

  1. Set an annual email marketing goal and strategy. Each year, compare results with the previous year’s results, so you know how to refine your strategy.
  2. Be flexible and agile. If a test fails, revamp and test around.
  3. Look beyond basic email metrics such as open rates, CTR, etc.
  4. Make use of the reports generated in the email campaign software for each campaign you plan.
  5. Make strategic reviews easier by gauging activities on a quarterly basis.
  6. Review Google Analytics, email metrics, and conversion in the same sitting for a holistic view.
  7. Move on once you have judged what works best for a particular element.
  8. While evaluating the results for proposing corrections—don’t put email aesthetics before results.

Share your best tip

What’s your best tip for optimizing email marketing results? Share in the comments below.

Read other Crazy Egg articles by Kathryn Aragon.



Get updates on new articles, webinars and other opportunities:

Kathryn Aragon

Kathryn Aragon is the former editor of The Daily Egg. She's a content strategist, consultant, and author of The Business Blog Handbook. Learn more at Follow her on Twitter.


Comment Policy

Please join the conversation! We like long and thoughtful communication.
Abrupt comments and gibberish will not be approved. Please, only use your real name, not your business name or keywords. We rarely allow links in your comment.
Finally, please use your favorite personal social media profile for the website field.


Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

  1. Katie says:
    May 29, 2015 at 1:12 pm

    Great infographic! I think the most important thing to remember is to test small – it can be something as simple as changing the colour of your CTA button that makes a world of a difference. Small, precise changes can completely change the outcome of any email campaign.

    • Kathryn Aragon says:
      May 29, 2015 at 5:25 pm

      Thanks, Katie! Great tip. I like your emphasis on small, precise changes.

  2. chawki trabelsi says:
    May 28, 2015 at 11:23 am

    Thank you for the tips Kathryn. My best tip for optimizing our email marketing campaigns is the pre-testing elements. It’s important to review old test results to know what design, email copy that doesn’t worked and then optimizing it for better conversion.

    • Kathryn Aragon says:
      May 28, 2015 at 11:48 am

      Great reminder, Chawki. Thanks for sharing!

Show Me My Heatmap

Set up some new tests today with - awesome tool, recommend for folk

Alex Peters


What makes people leave your website?