Is Bounce Rate Still A Good Deliverability Metric?

Uncategorized
Is Bounce Rate Still A Good Deliverability Metric?

A lot of people still use “Bounce Rate” as a way to measure things like Data Provider Quality and Email Deliverability.

But our two year long research study found that this metric is no longer as useful as it once was for measuring deliverability.

In this article, we’ll take a closer look at why this is the case, and what you can do to better measure email deliverability.

What is an “email bounce” anyway?

An email bounce is a notification that a sender receives letting them know that their email can’t be delivered with an explanation of why.

For example, in the notification below, we are being told that the targeted address does not exist.

Marketers try to avoid these types of bounces because they are very rare with legitimate email traffic and are usually a dead giveaway that someone is sending SPAM. They assume that spam filters will catch onto this very quickly… our question was, “is this true in 2024?”

Why We Became Suspicious Of Bounce Rates

We first started asking this question in the early days of Kitt AI when we solved catchall email verification. A cold email agency signed up and gave us some feedback:

User: “It’s not working – we sent an email to a valid address and it bounced”

Us: “How did you send the email?”

User: “We set up a test email account. Just forwarded the email to you”

Us: “This was sent from a gmail.com address with just ‘Hi’ in the subject line and no text. We just sent a proper email from a B2B gmail address and got no bounce back”

User: “…”

Then, just for fun, we decided to run a spam check on the email content and found that indeed, this type of message would get you spam listed automatically, which made us even more suspicious that the bounce message was “fake”. As you can tell from the image below, PYZOR (a service that scans email bodies to identify mass blasted emails that are showing up around the internet) flagged this particular email.

Before this experience, we had always assumed that bounces were a standard part of the email protocol and that we could rely on them to help us understand data quality and deliverability. However, several experiences like this led us to question whether that was true, so we decided to dig deeper.

How reliable are email bounce notifications?

To answer this question we took two approaches in our research.

First, we responded to all of the aforementioned customer complaints of bounced emails by sending emails to those addresses from various email providers and server configurations, and indeed confirmed that there were differences in behavior depending on the sender.

Second, we went deep into the documentation of several major email server and spam filtering systems, and we confirmed that the behavior behind bounce emails was completely configurable (as well as, in some cases bounce emails could be sent and fail to reach the target inbox)

The conclusion: a bounce email is a pretty meaningless metric without having additional context of which email systems you’re sending to, and what their configurable behaviors are. A good general rule is — the more customized the setup (eg: using an Email Security Gateway, using on-prem email servers, etc) and the more security oriented the target (financial services, government, higher education, etc), the less reliable we found these bounce notifications to be.

Sidenote: this is the type of exhaustive research and mappings that we had to do while working on Kitt AI, so you can be extra sure in the accuracy of our results!

Is “Bounce Rate” Becoming Less Reliable?

To answer this question, we need to first ask “how reliable was it in the first place?”, and then evaluate changes that may have happened in the landscape the past few years.

As we mentioned above, “bounce rates” can be quite reliable if you can view them in the context of the email servers that are sending the bounce (assuming you know their behaviors).

A long time ago, most email marketers followed two best-practices — first, only emailing SMTP verified addresses AND second — only emailing users who did a ‘double opt-in’ (first they submit their email address, then they confirm their subscription by clicking on a confirmation email link)

Bounce rates from such lists tended to be reliable because email servers that don’t implement any security (eg: catchall email addressed) tend not to have any intelligence in their bounce notifications, and bounces from double-opt-in emails can be clearly seen as ‘false’.

But the landscape has changed a lot over the past decade as we now see:

  • More “cold emails” to email addresses without verification/opt-ins
  • Increased use of email security software, designed to obfuscate the truth from cold emailers
  • Sender profiles that are fundamentally more ‘dirty’ (eg: containing signals that tie them to cold email software like Outreach.io, Salesloft, etc)

All of these factors have made bounce rate a lot less reliable.

So what can I do about this?

In a previous article, we shared alternate approaches towards measuring Email Deliverability, including our favorite one.

But if you’re the type of person who lives by the pareto principle (80% of the progress with 20% of the work), there are some simpler litmus tests you can use involving bounce rate.

Use A Better Email Verifier:

Part of the ‘doubt’ imposed today involves sending emails to ‘catchall verified’ emails. As we discuss in other articles, many of these verifiers check if an email was ‘once’ real, not if it’s currently real.

For example, Kitt AI’s identity verification feature is one of the most ‘sure’ way to know whether a bounce is “real” or not — meaning that if you get a bounce to one of these verified emails you can be 99% sure it was a fake bounce and that you have a deliverability problem against the target server.

Segment Your Bounce Rate Reporting

Outlook and Gmail make up >60% of the cloud b2b email market and have very consistent behaviors with bounce rates. By tracking the MX record of these bounced domains and confirming that they point to outlook.com or google.com, you can be fairly confident in the information they provide.

Conclusion

Bounce rates are similar to open rates, in that they are worth monitoring for directional changes, but are becoming increasingly “useless” as objective benchmarks. For further reading on this:

  • Check out our previous article on How To Measure Email Deliverability
  • Subscribe to our blog below to get the latest from our delivery lab research… we have lots of new Gmail related research coming out soon!

eroltoker

eroltoker

Author

Loading...