We build our mailing lists with only permission-based subscribers. We work on improving our mailing list opt-in rates. By knowing how recipients respond to our emails, we can fine-tune our efforts for the greatest results.

How to conduct email marketing tests

Set up a control group and a test group. With a large mailing list, the control group and the test group can each be segments of the larger group. After we see which email got better results, we can send that one out to the larger group.

Test each email element separately. When we send a control group one email and another group an email with a variation of only one feature, we can observe to what degree (if at all) people respond differently to that variation.

Change only one variable. For example, if we’re testing a new offer and send the email to the control group and the test group at different times, we won’t know if any response differences are because of the offer or the time difference.

How to measure results

These three factors determine levels of email campaign success:

• Open rate
• Click rate
• Conversion rate

Our goals are to get email recipients to open our emails, click on the links, and buy our products or services. For tests to be most effective, we should focus on improving just one of those goals with each test.

Open rates are tracked (in HTML email) with an image that’s loaded when recipients open emails. Because many email programs block images, however, the statistics for this image don’t accurately indicate how many emails are opened. But they can be useful in comparing what percentage of emails appear to be opened in different campaigns.

Click rates are tracked by the links we use in each email. Click tracking software makes it easy to track the effectiveness of email campaigns. We can also use different links for test groups and control groups and look in our site stats for the results.

Conversion rates depend on our goal. If our goal is to sell Product A, we can measure what percentage of people bought Product A after entering the site via the link or landing page in the email they received.

What email elements to test

Does the day and time sent change how people respond? Sometimes it does. If our recipients receive the emails at busy times for them, the emails may sit unread in in-boxes or get deleted without being opened. We want to find out if our audience responds more to email sent on Tuesday afternoon or Saturday morning.

We may want to test several different subject lines before starting a campaign. The better the subject line is, the more likely the email is to get opened.

After our readers open the email, do they react more positively if they see their name in it? We can find out by testing salutations.

If we’re using HTML email, the email design and layout needs to be as inviting as possible. Not too long or too short, too busy, too bright, or too dull. As with everything else, our readers’ opinions of these elements are subjective. We need to find what appeals to them.

Like subject lines, article headlines have to get people’s attention. If they don’t, we lose readers.

The landing page is another point where we have just a few seconds to convince readers to keep reading. The content, the layout, the colors, and everything about this page influences people to stay or not stay.

Every marketing email needs a call to action. Just one? Two or three? What placement and anchor text works best? How effective are image links compared to text links?

What offers bring more people to the website? Do time-limited offers work better or worse than some discounts? How big do discounts have to be to get people to buy? Are certain sale products more of a draw than others? Perhaps free shipping is a big draw — or not.

How to use the results

Most of the results are usually available within a few days after conducting the test.

Differences between the test group and the control group need to be statistically significant. And larger groups provide more meaningful data. But a test group of several thousand people or more has its drawbacks. We lose the opportunity to send a lot of people the more effective email if the test email fared more poorly than the one that the control group received.

If we find different results for different test groups, we may want to target segments of our mailing list, for example:

• New subscribers versus long-time subscribers
• Big spenders versus budget spenders
• More active purchasers versus less active ones
• Men versus women

Data collected from a number of tests provide comprehensive information about our audience. We can compile this information to better target our email list — not just once, but for all future mailings. That is, until we run more tests and fine-tune our approach even more.