Training Programs

Mastering Key Email Capture Terms and Definitions

How to calculate your email capture rate, understand A/B and sequential testing, and use a confidence calculator

Calculate your Email Capture Rate

Kim Mateus:
All right. So the webinar’s called 12 Ways to Grow Your Email List. And we decided to dedicate the first four ways to really more the conceptual level things that we think it’s important to understand a couple of key email capture terms and definitions that will form a nice baseline for you guys as you either begin your testing journey or are here to just look for additional ideas and ways to continue on your testing journey that you may have already be getting started on and executing on.

Kim Mateus:
So number one, we feel like it’s really critical to calculate your email capture rate. And the good news is that you guys all have one today and it’s just a matter of figuring out how to calculate it. So the two main things we look at on any given month for all of our clients and partners is how many external, new unique users did we get to the website this month and then how many net organic confirmed email subscribers have we added in this month?

Kim Mateus:
And the calculation of those two numbers then gives you an email capture rate. In this particular example, we’d be looking at a hundred thousand external, new unique users, we’re capturing 1800 net organic confirmed subscribers, giving us an email capture rate of 1.8%. And I know Bill for sure on that top number, we exclude traffic that’s already coming from email, right? Obviously if they’re coming from our email, then we know we’ve already captured them, so we definitely exclude that. Anything else to kind of keep in mind with how to get at these two numbers?

Bill Dugan:
Sure. That top number, just to delve a little deeper in addition to everything Kim described, does not include traffic that is driven from paid media sources, for example. What we’re talking about today is really about organic email capture rate. So this is not page views, this is not traffic coming from pay-per-click ads, and those are ways that you can also boost your email list, but this is about organic new site traffic that is unknown and not captured.

Bill Dugan:
And on the net confirmed, our best practice is a double opt-in. It’s a constant battle to try and manage the spam emails and spam traffic that can come to your site. Having double opt-in is one way, it’s not a perfect solution, but it certainly minimizes the extent to which you get random people signing up or bots signing up various email addresses on your site.

Bill Dugan:
So this confirmed to us means at least double opt-in. We have one client who requires a confirmation on clicking the link in the double opt-in process. So there are various ways to do this. But think of this whole thing that we’re talking about today as each element that we’re going to show you, each ad, each method is really a funnel, taking these new unique users to visit your site and driving them down a funnel with your ultimate goal to capture their email address.

Bill Dugan:
Because email is the linchpin of these kinds of systems, that’s how you’re going to communicate most of the time with customers that you do capture driving them back to the site, making offers to them, et cetera. So knowing what your baseline is, of what your email capture rate is a good starting point.

Kim Mateus:
Perfect. All right. The next two ways, take into consideration the different types of tests that you can run. I know, Bill, we execute on both of these. So the first one is AB testing. So do you want to talk to us a little bit about this and when you feel it’s appropriate to do an AB test versus the next way that we’re going to look at, which is sequential testing? But first help us understand AB.

Understand A/B Testing

Bill Dugan:
Sure. This is the most disciplined way to do a test and will give you the most accurate results. Just like the old days of direct mail, when you do an AB testing, you would isolate the cohorts in a random way so that you could run a test and get decent results. It’s the same thing here online, we’re taking a particular element and splitting serving of that ad randomly to an A audience and a B audience. And we tend to use this kind of testing when it’s really buttoned up the technology about how to do that splitting, and when you’re going to get decent volume so that you can get a statistically significant result.

Kim Mateus:
Perfect. And what about sequential testing?

Understand Sequential Testing

Bill Dugan:
Sequential testing, not as always as accurate as AB testing, because you’ve, first of all, got the differential of time. The first part of your test was run in this example in March and the next in April. So one thing to consider about sequential testing is the seasonality of your topic and your audience. You can’t control events.

Bill Dugan:
For example, had somebody done a sequential test last year between February and March or even March or April, the changing world of the pandemic could really get false results. So basically what you’re doing here, and another reason for doing a sequential test, is it could be very difficult technologically to execute an AB test because of the way that your site is constructed, because of multiple elements involved.

Bill Dugan:
And we’ll show you an example later of a sequential test that we’ve done. So what you’re isolating here is not an A and a B cohort at a single point in time, but you’re isolating the time. You run the test for the month of March. And another reason for doing a sequential test is you might need to run it for two months. You might do a March and April and then switch out all of your variants for May, June. And always be aware that those kinds of tests are going to be less accurate than a single point in time AB test.

Kim Mateus:
Let’s say for example, if you don’t have the kind of volumes needed to get a good AB result, this might be your only option really though.

Bill Dugan:
That’s right. Volume is another consideration.

Kim Mateus:
And I think that leads us to the next way, which is to use a confidence calculator. And we’ve included the link here of the one that we use. And I don’t know, Bill, I feel like we must use this across every test regardless of the volume?

Use a Confidence Calculator

Bill Dugan:
We do because built into the calculator and the confidence level is the volume and the variability. So we use the split test calculator decision tool on every test that we run. And what happens with volume challenged tests, so to speak, is that you have to run them longer.

Bill Dugan:
You might come and you think you’re going to run a test for a month, and the differential might be so small that you can’t declare a winner, or one is leading, but not by statistically significant an amount to declare a winner. So we’re constantly using this tool to quickly track the confidence level of a test. And we may decide that what we thought was going to be a month long test, even an AB test, has to run longer because you didn’t have a big enough differential to declare something significant.

Kim Mateus:
Awesome.

Bill Dugan:
In the old days we used to do this with a calculator or even with a spreadsheet, but this is a wonderful online tool that is free and fast.

Comments

Leave a Reply