Beyond Optimization: Email A/B Tests That Will Improve Your Entire Business

Those email metrics may provide you with more insight than you thought. Image via Shutterstock.

The components of an A/B test are pretty straightforward: change some stuff, compare key metrics, deploy winner, repeat.

So when you start an A/B test on your email, this is the sort of process you fall back on. You brainstorm a couple of alternate subject lines, test them on a small segment and send the winner to everyone else. This is a great way of making sure you're sending the better of two ideas, but does it really mean you're sending better email?

Instead, today we're going to focus on the benefits of A/B testing for the future. That means turning your results into actionable guidance for feature planning, branding, sales and retention strategies.

Maximizing is not always optimizing.

Feature planning

It can be really tough figuring out which features need the most attention, not to mention prioritizing improvements your top users would be most excited for. Email can help!

A simple email teasing upcoming improvements to X or Y feature can give you valuable insights for your next product planning session on what changes actually pique a user's interest.

Similarly, you can test something like, “What would you like to see added to feature X” vs “…feature Y.” Even if you get little to no feedback, the comparative open rates can tell you a lot about which features people want to see updated.

This can be especially insightful for startups, because setting the wrong priorities for your development team can hamstring your growth. In cases like this where the stakes are higher, it may be more powerful to subtly present options and observe responses than to straight up ask.

The problem with asking users what they want directly. Image via Frankiac.

Product branding

What if you're getting ready to launch a new feature or plan an event, but you're torn on what to call it. Simply run a test with a sneak peak email to your most engaged users and see what gets their attention.

This one may feel a bit weird, because branding of your product and features can feel really personal, but it's also really important, so why leave it to your gut when you can test?

You don't even have to build out a fancy announcement email, because you're just looking for opens, indicating that initial spark of interest. The body can be a simple, plain text save the date or a link to a survey or something.

For the rest of your emails...

Write a click-worthy subject line every time with our Subject Line Cheat Sheet.

By entering your email you'll receive weekly Unbounce Blog updates and other resources to help you become a marketing genius.

Sales materials development

Good email testing can also translate to benefits for your sales team. Imagine their eyes lighting up when you pass them a document illustrating how your highest value customers engage with different phrasings of your core features.

There are a couple of interesting ways to execute on this, but I think the most practical is to build an onboarding email with links to your features, and then test headlines for each section (bonus points if you randomize the order to satisfy the statisticians in the house).

You could also stretch this across multiple emails in your onboarding drip campaign, or send a one-off “What's new” update.

Life is about decisions. Image source.

Retention

Now that you've figured out which features resonate most with your high-touch users, it's time to figure out what gets people hooked on your product or service in the first place.

There are a ton of ways to accomplish this in the traditional on-site manner, but how does email fit into the picture?

The most obvious option here is to use the information you gleaned to craft a killer onboarding campaign that introduces new users to the most beloved features first. That strategy, however, is really focused on top-of-funnel retention. Today, I want to take a look at the other end: churn prevention.

There are, of course, some users that were never a good fit to begin with and will churn regardless. But for those that just never got the hang of things, the most common move is to hit them with a “Hail Mary email” - one last-ditch effort to win them back.

A lot of times this comes in the form of a direct note from someone asking what they could have done better, but why not use that space to run some tests? Not just to squeeze out a few more opens on a low-converting email, but to see what actually gets people's attention. Then you can take the stuff that works, and work it into your onboarding campaign to keep people from ever getting to the Hail Mary state.

Saying goodbye is hard. Image source.

Conclusion

These are, of course, not the only ways that you can incorporate your learnings from tests into other aspects of your marketing, but it's a great start if you don't have a process like this in place.

The structure you build out to track and share the results from tests like these can be tremendously helpful for the whole team - not just in the ways I've outlined above, but also in just keeping everyone on the same page and in line with what your customers want to hear.

Have anything to add? I'd love to hear about it in the comments.